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Executive Summary 



The purpose of the 2011 NAEP Writing Framework is to describe how the new NAEP 
Writing Assessment is designed to measure students’ writing at grades 4, 8, and 12. As 
the ongoing national indicator of the academic achievement of students in the United 
States, NAEP regularly collects information on representative samples of students in 
those three grades. 

The use of written language has become a critical component of the daily lives of 
millions of Americans. This is in part because, as technology continues to alter societies 
and cultures, it has fostered and supported an unprecedented expansion of human 
communication. In 2005, 172,000 new books were published in the United States alone. 
One hundred million websites existed worldwide. One hundred and seventy-one billion 
e-mail messages were sent daily. To write in this world is to engage in a millennia-old act 
that is reinventing and regenerating itself in the modem age. 

The impact of communications technologies has changed the way people write and the 
kinds of writing they do. Writing in the 21 st century is defined by its frequency and 
efficiency, and modern writers must express ideas in ways that enable them to 
communicate effectively to many audiences. It is clear that the ability to use written 
language to communicate with others — and the corresponding need for effective writing 
instruction and assessment — is more relevant than ever. 

Given expanding contexts for writing in the 21 st century, the 2011 NAEP Writing 
Framework is designed to support the assessment of writing as a purposeful act of 
thinking and expression used to accomplish many different goals. Although NAEP 
cannot assess all contexts for student writing, the results of the 201 1 NAEP Writing 
Assessment will offer new opportunities to understand students’ ability to make effective 
choices in relation to a specified purpose and audience for their writing in an “on- 
demand” writing situation. In addition, the assessment results will provide important 
information about the role and impact of new technologies on writing in K-12 education 
and the extent to which students at grade 12 are prepared to meet postsecondary 
expectations. 

The 2011 NAEP Writing Assessment reflects writing situations common to both 
academic and workplace settings, in which writers are often expected to respond to on- 
demand writing tasks. The 2011 NAEP Writing Assessment will assess ability to respond 
to a 30-minute writing task. Students at all three grades, including students with 
disabilities and English language learners, will complete two on-demand writing tasks. 

Development of this framework involved extensive research, outreach, and in-person 
meetings over the course of 18 months. More than 500 individuals from across the nation 
participated in the process. This framework reflects the results of extensive and thorough 
research on writing assessment (appendix F). Many resources were consulted, including 
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states’ writing standards and assessments, policy statements on writing assessment, and 
numerous journal articles. In addition, the framework reflects the perspectives of a 
diverse array of individuals and groups who collaborated on this project. These 
contributors included elementary, middle, secondary, and postsecondary educators; 
coordinators of writing instruction and assessment; experts in communication 
technologies; policymakers at the local, state, and national levels; representatives of the 
military; and business professionals. Members of many key professional organizations 
(appendix G) reviewed elements of the framework at various stages of development and 
provided their guidance. State testing and curriculum experts were consulted via in- 
person and computer conference sessions held throughout the framework project. This 
18-month effort has resulted in a rigorous and innovative new framework for the 201 1 
NAEP Writing Assessment. 

Key Features of the Framework 



Communicative Purposes and Rhetorical Flexibility 

On the 2011 NAEP Writing Assessment, students will have the flexibility to make 
rhetorical choices that help shape the development and organization of ideas and the 
language of their responses. Using age- and grade- appropriate writing tasks, the 
assessment will evaluate writers’ ability to achieve three purposes common to writing in 
school and in the workplace: to persuade; to explain; and to convey experience, real or 
imagined. Because understanding the nature of one’s audience is fundamental to 
successful communication, writing tasks will specify or clearly imply an audience, and 
writers will be asked to use approaches that effectively address that audience. 

Given the topic, purpose, and audience for the writing task, writers will be expected to 
draw upon a variety of approaches to thinking and writing in order to develop and 
organize their ideas and to craft language in ways that help them achieve their purpose 
for writing. At grades 8 and 12, students may be asked to choose the form or text type 
they wish to use in responding to the writing task — such as an editorial, a letter, an essay, 
etc. — that they believe best helps them achieve their purpose for writing. Writers’ work at 
all three grade levels will be evaluated using a holistic scoring guide that will support the 
assessment of how well all elements of a piece of writing work together. 

Computer-based Writing 

The design of the 2011 NAEP Writing Assessment reflects the way today’s students 
compose — and are expected to compose — particularly as they move into various 
postsecondary settings. The assessment is designed to measure the ability of students at 
grades 8 and 12 to write using word processing software with commonly available tools. 
Students will complete writing tasks by using their knowledge of effective uses of 
language in order to make use of common tools for editing, formatting, and text analysis. 
The framework encourages a computer-based assessment at grade 4 by 2019. 
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Profile of Student Writing 



The 2011 NAEP Writing Assessment may introduce a new component to reporting that 
will provide a more detailed survey of writing achievement at grades 4, 8, and 12. This 
Profile of Student Writing will provide a deeper analysis of more specific dimensions of 
students’ responses, particularly with regard to how writers have approached the develop- 
ment and organization of ideas and to how they have used language in relationship to 
each communicative purpose being assessed. 

Plans for 2011, 2013, and Future Assessments 



In May 2010, the National Assessment Governing Board made several changes to the 
NAEP Schedule of Assessments, including a delay of the grade 4 NAEP Writing 
Assessment to 2013. The plan is to conduct a computer-based assessment at grade 4, 
which will enable NAEP to collect baseline data on fourth graders’ computer-based 
writing skills. This change is also consistent with the Board’s goal to move all NAEP 
assessments to a computer-based platform in the future. In both the 201 1 NAEP Writing 
Assessment at grades 8 and 12 and the 2013 NAEP Writing Assessment at grade 4, 

NAEP will report achievement results for a nationally representative sample of students. 
As computers become more prevalent in schools, reporting state-level results and results 
for districts in the NAEP Trial Urban District Assessment may also become feasible. 

As in the past, the depth and extent of the information available from the results of the 
assessment will provide important data on writing achievement in 201 1 and beyond. The 
assessment results reported by NAEP will provide the public, policymakers, and 
educators with important new information about the achievement of student writers and 
the nature of their performance in different communicative situations. There are, 
however, limitations to the range and scope of skills that NAEP can assess because, like 
most standardized assessments, NAEP is an “on-demand” assessment with limited time 
and resources. Therefore, the assessment results in 201 1 and beyond should not be 
interpreted as a complete representation of student writing performance. 

Results of the assessment will also be reported in The Nation’s Report Card by means of 
scale scores and achievement levels, which present the percentage of students who 
perform at the Basic, Proficient, and Advanced levels. Demographic and subject-specific 
data gathered from students, teachers, and schools will also be available. While student 
performance trends in writing were reported from 1998 to 2007, the resulting information 
from the 201 1 NAEP Writing Assessment will be used to establish a new trend line that 
should continue for at least ten years. 
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Chapter One 



Overview 

The purpose of this chapter is to present an overview of the 2011 NAEP Writing 
Assessment. 

Introduction 



The ability to write well is essential to the economic success of the nation. Americans in 
the 21 st century need to be able to communicate in a variety of forms and mediums, 
create texts under the constraints of time, and play a productive role in an economy that 
increasingly values knowledge and information. The pace of written communication in 
today’s environment — the velocity of writing — reflects the transition to an information- 
based economy built on speed, efficiency, and complexity. 

Writing is essential to productivity and to personal and social advancement. Corporations 
in almost all industries and services report that a significant majority of salaried 
employees — 80 percent or more — have some responsibility for writing in their 
professions, a substantial growth from previous decades (Berman, 2001). It is no wonder, 
then, that communicating effectively is considered the most desirable skill among new 
hires by major corporations, and that good writing is essential to mid-career professionals 
for both day-to-day operations and long-term career advancement (National Association 
of Colleges & Employers, 2005; Light, 2001). 

Developments in the theory and practice of composition underscore these changes and 
provide the basis for the 2011 NAEP Writing Framework. Researchers, conscious of how 
social influences shape the composing process, define writing as an action shaped by its 
intended goal and by the expectations of a reader (Miller, 1984). Writing, then, can be 
understood to be a negotiation between the demands of a writing situation — its 
objectives, its audience, etc. — and what one decides to act upon in order to complete the 
writing task and satisfy the demands of those reading it. Clarity of expression and 
effective presentation of ideas depend on a writer’s ability to focus and organize 
information and to correctly employ conventions of language. 

In K-12 education, good writing instruction empowers students to acquire new 
knowledge and to develop critical thinking skills. This is true of writing in all subject 
areas, not just English language arts. Writing and reasoning effectively are increasingly 
embedded in the learning of every subject discipline (Squire, 1988). Moreover, writing is 
not merely a school-based practice but a lifelong skill used to accomplish specific goals 
and convey particular messages within community and workplace settings. 

The use of computers — in the workplace, in schools, and in the home — has also reshaped 
the nature of writing and the importance of effective communication. Government studies 
have shown that as many as 96 percent of K-12 students use computers to some degree 
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for academic or personal purposes (U.S. Department of Commerce, 2002; DeBell & 
Chapman, 2006). Similarly, a Bureau of Labor Statistics (2005) survey reports that more 
than 90 percent of college graduates use computers at work. The increasing frequency of 
computer-based writing outside of school or the workplace (e.g., e-mails, instant 
messaging, blogs) will undoubtedly expand the variety of writing situations in the future 
to include many new purposes and audiences. Future trends in writing instruction must 
take into account how computers affect both the writing process and the types of text 
produced. 

The computerization of the 2011 NAEP Writing Assessment takes into consideration the 
prevalence of computer technology in schools and the workplace, the projected future 
growth of large-scale computer-based tests, and the increasing role computers play in the 
economic and educational activities of the nation. Initial research on the effects of 
computer-based writing in the K-12 curriculum is promising. Several studies suggest that 
using word processing applications can lead to more collaboration with other writers, 
support the production of longer compositions, and encourage the use of researched 
arguments that require inquiry and investigation (Baker & Kinzer, 1998; Goldberg, 
Russell & Cook, 2003; Graham & Perin, 2007; Grejda & Hannafin, 1992, Lunsford & 
Lunsford, 2007). 

Key Goals for the 2011 NAEP Writing Assessment 

Though the number of large-scale direct writing assessments has increased since the 1998 
NAEP Writing Framework was developed — particularly with the incorporation of a 
writing component into the ACT and SAT college admissions tests — NAEP continues to 
provide the only nationally representative data on writing achievement. As a survey of 
student achievement, NAEP items are designed to measure what students know and are 
able to do in relation to the instruction they have received and in relation to expectations 
for postsecondary preparedness. To that end, the 201 1 NAEP Writing Framework has the 
following goals: 

To encourage student writers to move beyond prescriptive or formulaic approaches in 
their writing. In this framework, the decisions writers make about how to develop and 
organize ideas and use language are considered an important component of their writing 
ability. The most successful writers are those who are able to consistently make effective 
decisions in all dimensions of their writing in order to communicate effectively with their 
audience. Therefore, writing tasks in this assessment will be designed to support the 
evaluation of students’ ability to make a variety of effective choices in how they 
approach the development and organization of ideas and in how they craft language to 
support their communicative purpose. 

To assess students ’ writing using word processing software with commonly available 
tools. Because the computer plays a significant role in writing production, the technology 
used to compose is an important part of the writing process and reflects new conditions 
for writing at school and at work. Thus, in 2011, at grades 8 and 12, NAEP will assess 
writing with word processing software. At grade 4, students’ keyboarding expertise and 
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experience using computers on assessments will be reviewed as part of the planning 
process for the NAEP computer-based Writing Assessment for fourth grade students 
scheduled in 2013. 

To measure students’ ability to respond to a writing task in an on-demand scenario. 
While writing tasks in schools or the workplace often involve composing and editing 
processes lasting days or weeks, on-demand writing situations occur in both academic 
and professional settings where writers must often compose to achieve goals under time 
constraints. The 201 1 NAEP Writing Framework is designed to provide information 
about what students can accomplish in such on-demand writing situations. 

Writing in the Context of NAEP 

Writing is a complex, multifaceted and purposeful act of communication that is 
accomplished in a variety of environments, under various constraints of time, and 
with a variety of language resources and technological tools. 

People communicate to accomplish goals or meet needs. Writing, then, can be thought of 
as a relationship or negotiation between the writer and reader to satisfy the aims of both 
parties. As a result, the 201 1 NAEP Writing Framework focuses on writing for 
communicative purposes and on the relationship of the writer to his or her intended 
audience. The 2011 NAEP Writing Assessment measures three communicative purposes 
common to academic and professional settings: 

In a complex society with a plurality of perspectives and opinions, students need to be 
capable of expressing their viewpoints clearly and logically to convince others. 

Therefore, the 2011 NAEP Writing Assessment measures the ability to persuade, in 
order to change the reader’s point of view or affect the reader’s action. 

The ability to inform others of ideas and concepts is also critical in an information-driven 
society. Therefore, another communicative purpose measured by the 2011 NAEP Writing 
Assessment is the ability to explain, in order to expand the reader’s understanding. 

Finally, in an era in which many of the borders that have long separated the world’s 
peoples blur, exploring and sharing human experience through writing helps define not 
only individual identity but also the universal connections that people share. Therefore, 
the 201 1 NAEP Writing Assessment also measures writers’ ability to convey experience, 
real or imagined, in order to communicate individual and imagined experience to 
others. 

In choosing to evaluate these purposes for writing, developers of the 2011 NAEP Writing 
Framework do not intend to discount the importance of other common purposes for 
writing. K-12 curricula are rich with writing experiences in all subject areas. Many 
writing situations encourage students to write as a means of self-expression and 
comprehension, as is the case with writing-to-learn activities when the student composes 
as a means of thinking through key ideas on a topic. The importance of written 
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communication for personal purposes cannot be overstated: students given adequate 
practice in developing their own thoughts and feelings through such writing are better 
able to perform well in all forms of writing (Britton, Burgess, Martin, McLeod & Rosen, 
1975). While NAEP cannot assess personal writing tasks (e.g., journals) or longer 
assignments (e.g., research reports or multimedia projects), it can provide a national 
survey of writing for these three communicative purposes in an on-demand writing 
situation. 

For a glossary of terms used in the 2011 NAEP Writing Framework, see appendix A. 

Content of the 2011 NAEP Writing Assessment 



Writing tasks for the 201 1 NAEP Writing Assessment will allow students to include 
information and ideas from their own reading, observations, and experiences, or to 
respond to short reading passages or to visual stimuli such as photographs or simple 
visual displays of information. The NAEP Writing Assessment will measure student 
ability to write in English. At all three grade levels, age- and grade-appropriate reading 
passages and visual stimuli will be incorporated as resources in some writing tasks. 

Writing tasks will be as open-ended as is appropriate to allow students to use supporting 
ideas that best fit the purpose and audience for the writing task. Tasks will not be subject- 
area specific; if they choose, students will be able to respond with ideas from areas 
outside of English language arts, such as history and science. The complexity of the tasks 
and the level of sophistication expected in the responses will increase at each grade level. 

Examples of writing tasks at grades 4, 8, and 12 are provided in chapter two and in 
appendix B. 

The Role of Purpose 

Throughout K-12 and higher education, as well as in the workplace, most required 
writing falls under the broad categories of persuasive texts; explanations of events and 
phenomena; and narratives and reflective pieces, both real and imagined. Thus, the 201 1 
NAEP Writing Framework assesses these three communicative purposes: to persuade, to 
explain, and to convey experience, real or imagined. Tasks for the 2011 NAEP Writing 
Assessment will clearly identify the purpose for writing, providing NAEP with an 
important context for assessing student writing achievement at grades 4, 8, and 12. 

The Role of Audience 

Writing is a social act — not only do writers always write for a purpose, but they usually 
write to communicate ideas to others. Demonstrating an awareness of audience is 
considered to be one of the most important writing skills, particularly by college 
instructors and business professionals (Harris, 2006). 
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In most school and postsecondary writing situations, writers are either assigned an 
audience or the audience is clearly implied by the nature of the task. Therefore, tasks on 
the 2011 NAEP Writing Assessment will specify or clearly imply an audience for the 
writing. These audiences will be realistic audiences that correspond to the purpose of the 
writing task. Audiences specified on the assessment will generally vary for each grade. 

At grade 4, audiences may include peers, teachers and school officials, and parents; at 
grades 8 and 12, writers may more often be asked to write for less familiar, more 
authoritative audiences, such as school or community leaders and government officials. 

The Role of Approaches to Thinking and Writing 

When given a purpose and audience for writing, writers must decide how to develop and 
organize their ideas to achieve the demands of the task. Defined by various composition 
theorists as thinking and writing approaches or problem-solving strategies, such 
techniques allow writers to develop responses of depth and substance (Claggett, 2005; 
National Writing Project and Nagin, 2003; Flower, 1993). Some approaches commonly 
used to develop and organize ideas in effective written communication include analyzing, 
describing, evaluating, and narrating. By using these and other approaches to thinking 
and writing, alone and in combination, writers have considerable flexibility for the 
development and organization of a text. 

While writing tasks on the 2011 NAEP Writing Assessment will not specify the use of 
particular approaches to thinking and writing, tasks will be designed to encourage 
students to draw upon a wide variety of approaches to support the development and 
organization of ideas. Responses will be evaluated for the effectiveness of writers’ 
development and organization of ideas in relation to purpose and audience. 

The Role of Form in the 2011 NAEP Writing Assessment 

The term “form” refers to text features characteristic of a particular kind of writing, such 
as a short story or a newspaper editorial. The 2011 NAEP Writing Framework may call 
for students at grades 8 and 12 to choose the form they believe is most effective for the 
purpose and audience specified in the task. 

However, because the impact of asking students to choose a form in an on-demand 
writing task is unknown, three kinds of writing tasks will be considered during assess- 
ment development at grades 8 and 12: with the specification of form, without the 
specification of form, and with suggestions for two or more possible forms. The 
information from the assessment development process will help determine which tasks, if 
any, require the specification of form on the 2011 NAEP Writing Assessment. 

However, students at grade 4 will be assigned a specific form in each writing task. 
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Summary of Guidelines for Writing Tasks 



The key guidelines for writing tasks are summarized in Exhibit 1 . 1 and presented as a 
graph in chapter two (see Exhibit 2.1). 



Exhibit 1.1. Key Guidelines for Writing Tasks 





Key Characteristics 


Topic 


Topics will: 

• Address real-world, age-appropriate, and grade-appropriate 
issues. 

• Be familiar and accessible to students, and not controversial 
in nature. 

• Encourage the use of effective approaches to thinking and 
writing. 


Purpose 


Purpose will be: 

• Clearly stated in the writing task. 

• Age-appropriate and grade-appropriate. 

• Consistent with the audience identified in the writing task. 

• Distributed appropriately at all three grades. 


Audience 


Audience will be: 

• Specified or clearly implied by the context of the writing 
task. 

• Age-appropriate and grade-appropriate. 

• Familiar to students. 

• Consistent with the purpose identified in the writing task. 


Content 


Students will: 

• Make choices within parameters provided by the writing 
task (e.g., “persuade a classmate to read your favorite book” 
- writer would choose the book to write about). 

• Draw upon their experiences and observations. 

• Occasionally respond to an external stimulus, such as a brief 
reading passage or an illustration, photograph, table, chart, 
or other visual representation. 


Approaches 
to Thinking 
and Writing 


Students will: 

• Consider the purpose and audience for their writing task 
when determining how to develop and organize ideas and 
how to craft language. 

• Decide for themselves which thinking/writing approaches to 
use in developing and organizing ideas. 


Form 


Students may: 

• Choose the form most suitable to their purpose and audience 
at grades 8 and 12 (to be field tested prior to the 201 1 
assessment). Grade 4 students will be asked to respond by 
using a specific form. 
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Design of the 2011 NAEP Writing Assessment 



Writing with Computers 

In 2011 and beyond, the NAEP Writing Assessment at grades 8 and 12 will assess 
computer-based writing. For the purposes of the 201 1 NAEP Writing Assessment, 
“computer-based writing” means that students compose and construct their responses 
using word processing software on a computer, with the option to use commonly 
available tools. 

At grade 4, a computer-based assessment was previously impractical because of time 
constraints for computer instruction, the unequal availability of technology in elementary 
schools, and elementary school students’ current limited keyboarding proficiency. 
However, it is recommended that the assessment at grade 4 become computer-based 
during the tenure of the 2011 NAEP Writing Framework. It is expected that, by 2019, 
widespread access to computers and increased keyboarding instruction in elementary 
schools will provide students with more opportunities to compose on the computer, thus 
warranting the delivery of a computer-based assessment at grade 4. 

Rationale for Computer-based Assessment of Writing 

The movement to a computer-based assessment for the 201 1 NAEP Writing Assessment 
reflects radical changes in the uses of technology for writing since the development of the 
1998 NAEP Writing Framework. The number of students ages 12-17 who regularly or 
occasionally use computers to complete school work and to access the Internet has 
increased from 32 percent in 1995 to a range in 2006 between 87 and 93 percent (Pew 
Research Center, 1995; Hitlin & Rainie, 2005; DeBell & Chapman, 2006). During the 
tenure of the previous NAEP Writing Framework, the ratio of students-to-computer 
decreased from six students for every one computer in 1998 to 3.8 students for every 
computer in 2005 (Goldberg, et al., 2003; Technology Counts, 2006). In addition to an 
increase in the number of computers available to students, the use of computers to write 
has become nearly universal: 97 percent of grade 8 teachers surveyed on the 1998 and 
2002 NAEP Writing Assessments indicated that their students used computers to 
compose (Solomon, Lutkus, Kaplan, and Skolnick, 2004). These teachers saw the use of 
computers during the composing process as positively affecting student writing 
achievement, particularly with regard to increases in student motivation and time spent 
on revising written drafts (Solomon, et al., 2004). 

As computers are increasingly becoming the established mode for completing academic 
and professional tasks, computer-based testing is increasingly becoming the established 
mode for large-scale assessments. In 2006, half of all states assessed student learning on 
computers or were piloting computer-based assessments, and several other states outlined 
plans to transition to computer-based tests. By 201 1, a majority of states will likely offer 
some form of testing via computers (ACT Research Brief, 2006). In 2007, 19 states were 
using, piloting, or developing direct writing assessment using computers. Forty-seven 
states have separate standards for technology use; the remaining states have embedded 
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technology expectations in their subject area standards (ACT Research Brief, 2006; 
Russell & O’Connor, 2003). 

Concerns about students’ access to computers and equity of access are common among 
policymakers and educators. However, progress continues to be made in bridging the 
“digital divide.” The 2005 computer-to- student ratio at high-poverty and high-minority 
schools was only slightly higher than the national average of 3.8 — at 3.9 for high-poverty 
schools and 4.1 for high-minority schools ( Technology Counts , 2006). Expectations for 
student computer proficiency remain consistent across all educational contexts. The Trial 
Urban Districts that participated in the 2007 NAEP Writing Assessment all possessed 
benchmarks for computer proficiency — some developed their own, others followed state 
standards — that call for competence at grades 8 and 12 in producing written texts on 
computers and utilizing electronic resources. In several studies, including a 2002 NAEP 
pilot study examining computer-based writing assessment, most research has found that 
no significant differences exist in performance within subgroups (gender, race, socio- 
economic, etc.) when paper and pencil scores were compared to an online assessment 
(Horkay, et al., 2006; Russell, Higgins & Hoffman, 2004; Sandene, et al., 2005). 

Although there may still be students who will report not using computers for writing in 
201 1, it is reasonable to expect that the number of students who commonly use 
computers to write will substantially increase. Indeed, though the implementation of a 
computer-based writing assessment at grades 8 and 12 in 2011 may raise equity issues for 
those students who are not comfortable with electronic composition, a paper and pencil 
assessment would create similar issues of bias for students who commonly use computers 
to write (Russell, et al., 2004). Research has shown that paper and pencil assessments 
negatively affect the writing performance of computer-proficient students (Carlson, 2000; 
Russell, et al., 2004; Russell, 1999; Sandene, et al., 2005). 

Assessing Computer-based Writing 

The decision to assess computer-based writing at grades 8 and 12 addresses expectations 
for what students should know and be able to do at those grade levels. Because the ability 
to use word processing software and its tools has become a critical component of the 
composing process, the 2011 NAEP Writing Framework supports an assessment environ- 
ment for writing more similar to that used by students and adults who write on computers 
in postsecondary education, in the workplace, and in their daily lives. The goal of this 
assessment, then, is to measure writing achievement using word processing software with 
commonly available tools. 

In the context of NAEP, “tools” refers to the extensions built into word processing 
software that help writers modify or revise their written documents. For the 2011 NAEP 
Writing Assessment, the following kinds of tools will be considered: prewriting tools 
(e.g., notepad windows, paper); editing tools (e.g., cut, copy, paste); formatting tools 
(e.g., line spacing, indenting); spelling tools (e.g., spell check, automatic capitalization); 
grammar tools (e.g., grammar check); and reference tools (e.g., dictionary, thesaurus). 

The specific tools to be used will be determined at a later time, pending changes in 
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technology. Features considered distracting or irrelevant to what is being assessed will 
not be available to students, nor will the Internet or other online applications (e.g., 
databases, encyclopedias, etc.). 

Framework developers considered whether to enable any composing and editing tools. 
However, eliminating access to common word processing tools on the computer would 
create a highly artificial platform for composing, since a writer normally has access to 
and uses at least some common tools when composing on a computer. The purpose of 
assessing writing produced on the computer comes into question when access to such 
common features of word processing software is eliminated. 

Time per Task 

NAEP ascertains what students know and can do in a limited amount of time with limited 
resources (e.g., with limited opportunities for reflection and revision or for feedback from 
peers and teachers). The NAEP Writing Assessment assigns students two on-demand 
writing tasks. Each task will represent one of the three communicative purposes NAEP 
writing assesses — To Persuade ; To Explain ; or To Convey Experience, Real or Imagined. 
The writing assessment will be administered as two 30-minute, computer-based writing 
tasks. 

Distribution of Communicative Purposes 

Exhibit 1.2 shows the distribution of communicative purposes on the 201 1 NAEP 
Writing Assessment. The percentages represent the proportion of tasks for a particular 
purpose out of the total number of tasks developed for each grade. 



Exhibit 1.2. Percentage Distribution of Communicative Purposes by Grade 





Grade 4 


Grade 8 


Grade 12 


To Persuade 


30% 


35% 


40% 


To Explain 


35% 


35% 


40% 


To Convey 
Experience 


35% 


30% 


20% 



There is a progression of emphasis on writing to explain and to persuade, though many 
students in all three grades will also write to convey experience, real or imagined. The 
distribution of percentages reflects the focus on these three communicative purposes in 
writing instruction at the elementary and secondary levels, as well as the writing 
expectations in postsecondary settings. 

Assessing Students with Special Needs 

The NAEP Writing Assessment is designed to measure the writing skills and academic 
achievement of students at grades 4, 8, and 12, so tasks on the NAEP Writing Assessment 
will be as fully and clearly explained as possible in order to be accessible to all students. 
Further, students with disabilities and English language learners are included in the 
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assessment sample. The 2011 NAEP Writing Assessment will be administered to English 
language learners and students with disabilities who, based on inclusion criteria specified 
by NAEP, are capable of participating. 

Some students may need accommodations to be able to participate in the NAEP Writing 
Assessment. NAEP attempts to provide accommodations to students that match the 
accommodations these students are allowed in school (for example, a student would be 
offered the same accommodations as directed by the student’s Individualized Education 
Program), as long as those accommodations do not alter the construct being measured on 
the assessment. Appropriate accommodations will be available for computer-based 
delivery of the 2011 NAEP Writing Assessment. 

Evaluation of Responses for the 2011 NAEP Writing Assessment 



Which Features of Writing Will be Evaluated 

Three broad features of writing will be evaluated in students’ responses: 

• Development of Ideas 

• Organization of Ideas 

• Language Facility and Conventions 

These broad features are consistent with state learning standards and reflect what most 
states evaluate in their direct writing assessments at grades 4, 8, and 12. They are also 
consistent with expectations for postsecondary preparedness. 

Exhibit 1.3 presents these three broad domains and the important features within each 
domain that will be used to determine the level of achievement in students’ writing. The 
effectiveness of these features will be evaluated in relation to the purpose and audience 
specified in the writing task. 
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Exhibit 1.3. Criteria for Evaluating Responses 

Development of ideas is effective in relation to the writer’s purpose and audience. 

• Depth and complexity 

• Approaches to thinking and writing 

• Details and examples 

Organization is logical in relation to the writer’s purpose and audience. 

• Text structure 

• Coherence 

• Focus 

Language facility and conventions support clarity of expression and the effectiveness of 
the writing in relation to the writer’s purpose and audience. 

• Sentence structure and sentence variety 

• Word choice 

• Voice and tone 

• Grammar, usage, and mechanics (capitalization, punctuation, and spelling) 



How Responses Will Be Evaluated 

The evaluation of responses will be based on the criteria listed in the three broad domains 
above. The recommended tool for evaluation is a holistic scoring rubric, which is 
designed to guide scorers’ evaluation of the response as a whole using the criteria 
specified for the assessment. Individual elements or parts of the response will not be 
scored separately. Scoring rubrics will provide guidelines for determining the overall 
performance of a response on a scale from 1 to 6, with 1 being low and 6 being high. 
Trained readers will decide on a single score (one number on the scale of 1 to 6) based on 
a judgment about the relative strengths and weaknesses of the response in relation to 
specified criteria. 

Appendix C includes preliminary holistic scoring rubrics for each of the three 
communicative purposes to be assessed. 

Reporting Results of the 2011 NAEP Writing Assessment 



NAEP provides an ongoing representative sample survey of student achievement by 
administering assessments at regular intervals to students at grades 4, 8, and 12. In 
addition to reporting overall results at the national level, NAEP reports results at the state 
and jurisdiction levels, as well for large urban school districts participating in the NAEP 
Trial Urban District Assessment (TUDA). NAEP also reports on the performance of 
various student subgroups at the national, state, and urban district levels. 

The 2011 NAEP Writing Assessment will begin a new trend line for writing, which 
should continue for at least 10 years. The results of the 201 1 NAEP Writing Assessment 
will offer new opportunities to examine students’ ability to write effectively in relation to 
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the purpose and audience for the writing and to understand the role and impact of 
computers on writing production. 

NAEP Reports 

The primary means for public release of NAEP results is a printed summary report 
known as The Nation’s Report Card. This report is accompanied by a dedicated website: 
http://nationsreportcard.gov. The printed report and website will provide detailed 
information on the results of the 2011 NAEP Writing Assessment and the students who 
participated. Results will be reported for specific groups of students and for states and 
large urban districts that participate in the NAEP assessment. NAEP results are reported 
for demographic subgroups, including females and males, students from various 
racial/ethnic backgrounds, and students who took the 2011 NAEP Writing Assessment 
with and without accommodations. Individual student performance on NAEP 
assessments is not reported. 

The information available from the results of the 201 1 NAEP Writing Assessment will 
provide important new data during the tenure of the framework. Because the assessment 
measures student writing achievement within a specific context that is limited by time 
and resources, there are limitations to the data reported and to uses of those data. NAEP 
reports do not evaluate results or provide conclusive statements about the level of writing 
achievement among K-12 students, nor is the assessment designed to drive curriculum or 
writing instruction. 

Reporting Scale Scores and Percentiles 

NAEP writing results are reported in two ways: as scale scores and as percentages of 
students attaining each of the achievement levels. Scale scores, which are derived from 
student responses to assessment items, summarize the overall level of performance attained 
by a group of students. Scale scores are presented as average scale scores and as scale 
scores at selected percentiles. Achievement levels provide further information about 
student achievement by indicating the degree to which student performance meets the 
standards set for what students should know and be able to do. 

Achievement Levels and Cut Scores 

NAEP achievement levels are the primary means of reporting NAEP results to the 
general public and policymakers. Achievement Level Descriptions represent an informed 
judgment of “how good is good enough” in the various subjects assessed. NAEP 
achievement levels define in general terms what students at grades 4, 8, and 12 should 
know and be able to do at the Basic, Proficient, and Advanced levels. Cut scores 
represent the minimum score required for performance at each NAEP achievement level 
and are usually reported along with the percentage of students who scored at or above the 
specified level. Sample student responses provide illustrations of student skills within 
each level of achievement. 
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Exhibit 1.4 displays the Governing Board’s generic policy definitions for Basic, Proficient, 
and Advanced achievement that pertain to all NAEP subjects and grades. 



Exhibil 


1.4. Generic Achievement Level Policy Definitions for NAEP 


Achievement 

Level 


Definition 


Advanced 


This level signifies superior performance. 


Proficient 


This level represents solid academic performance for each grade assessed. 
Students reaching this level have demonstrated competency over challenging 
subject matter, including subject-matter knowledge, application of such 
knowledge to real-world situations, and analytical skills appropriate to the 
subject matter. 


Basic 


This level denotes partial mastery of prerequisite knowledge and skills that 
are fundamental for proficient work at each grade. 



The full text of preliminary Achievement Level Descriptions for the 2011 NAEP Writing 
Framework is located in appendix D. 

Background Variables 

Students participating in the NAEP Writing Assessment respond to questionnaires designed 
to gather information on variables important to understanding writing achievement 
nationwide. Teachers and school administrators also complete questionnaires to gather data 
relevant to student achievement. This information is used as part of the summative report 
on writing achievement. 

Questionnaires for students on the NAEP assessments will contain questions about 
demographics, learning habits or attitudes, and reactions to the NAEP Writing Assessment 
itself. Teachers and school administrators will be asked questions on these topics as well as 
others related to instructional practices, professional development, and teacher 
qualifications. Background questionnaires may also contain subject-specific questions that 
elicit more information from students and school personnel about different elements of the 
content areas. For the purposes of the 2011 NAEP Writing Assessment, the questionnaires 
will reflect two primary interests: computer use (particularly with regard to writing with 
computers) and the number and kinds of opportunities students have to write. The 2011 
NAEP Writing Framework includes some highlights of these recommendations in chapter 
five. 

Recommendations for background variables for the 201 1 NAEP Writing Assessment are 
presented in Writing Background Variables Recommendations . 

New Component of NAEP Reporting: Profile of Student Writing 

Framework developers recognize that more information about the meaning of the data 
gathered from the 201 1 NAEP Writing Assessment will be useful to K-12 schooling and in 
various postsecondary settings. The 2011 NAEP Writing Framework therefore 
recommends that a national sample of student responses at grades 4, 8, and 12 be selected 
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and examined to obtain more in-depth information on key features of student performance 
in writing. By selecting a nationally representative sample, the results of the analyses can 
be generalized to the whole population, supporting external validity of the conclusions. 
Four studies may be included in the Profile. 

• Analysis of Development of Ideas: The decisions students make when developing 
ideas — particularly those that relate to the communicative purposes assessed; the 
rhetorical flexibility offered in the tasks; and the criteria used to evaluate the 
development of ideas — will be analyzed. 

• Analysis of Organization of Ideas: Students’ methods of organization, the 
structures of their texts, and their use of forms in relation to communicative 
purposes will be analyzed. 

• Analysis of Language Facility: Collection of data on students’ language use and 
stylistic choices, common errors in the use of conventions, and students’ level of 
language complexity (e.g., word choice, sentence length) will be analyzed in 
relation to student performance data and background variable information. 

• Exploratory Analysis: Data produced by these three studies will be analyzed in 
relation to performance and in relation to information collected on the background 
variables. These analyses will address what patterns, if any, exist between 
observations of various dimensions of writing and the quality of the responses. 

The results of this investigation should be reported for writing done within the context of 
an on-demand writing assessment and at the national level only. The Profile will provide 
the public, policymakers, and educators with important new trend data beginning in 2011. 

NAEP Writing Special Study 



The transition to a computer-based delivery by 2019 for grade 4 will require further study 
before such a recommendation can be implemented. A special study will be needed prior to 
2019 to investigate changes in computer instruction at the fourth grade, on fourth graders’ 
keyboarding proficiency, and on the extent to which fourth graders are accustomed to 
composing using word processing software. 

Appendix E includes further discussion about the design of the special study. 

Comparison of the 1998 and the 2011 NAEP Writing Frameworks 



The 2011 NAEP Writing Assessment Framework replaces the framework developed for 
the 1998, 2002, and 2007 NAEP Writing Assessments. The 2011 NAEP Writing 
Framework honors many aspects of the previous framework but also introduces important 
changes that will lead to better measurement of student writing and more precise reporting 
of assessment results. 
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The chart on the following pages highlights key differences between the 1998 — 2007 and 
2011 NAEP Writing Frameworks. 



Exhibit 1.5. Comparison of 1998 — 2007 and 2011 NAEP Writing Frameworks 



1998—2007 Writing Framework 



The 1998-2007 NAEP Writing 
Assessments measured three 
modes: 



2011 Writing Framework 



201 1 NAEP Writing Assessment 
measures three communicative 
purposes: 
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Persuasive mode: 

o Writing to convince 

o Writing to construct an 

argument 

o Writing to refute a position 
Informative mode: 
o Description 
o Explanation 
o Analysis 
Narrative mode: 

o First-person and third- 
person fictional stories, 
personal essays 



• To Persuade, in order to 
change the reader’s point 
of view or affect the 
reader’s action 

• To Explain, in order to 
expand the reader’s 
understanding 

• To Convey Experience, 
real or imagined, in order 
to communicate individual 
and imagined experience 
to others 



On the 1998-2007 NAEP Writing 
Assessments, some writing tasks 
required students to write for a 
particular audience (e.g., a peer, 
school principal, or committee). For 
other writing tasks, an audience was 
not specified. 



In 201 1 , a specific audience will 
stated or clearly implied in all 
writing tasks at grades 4, 8, and 
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Grade 

4 


Grade 

8 


Grade 

12 


Persuasive 


25% 


33% 


40% 


Informative 


35% 


33% 


35% 


Narrative 


40% 


33% 


25% 





Grade 

4 


Grade 

8 


Grade 

12 


To Persuade 


30% 


35% 


40% 


To Explain 


35% 


35% 


40% 


To Convey 
Experience 


35% 


30% 


20% 
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Explanation for Change 



Purposes for writing are 

emphasized as a way of: 

• Recognizing that most 
writing is influenced in 
significant ways by 
interaction between writer, 
purpose, audience, and 
topic. 

• Focusing the writer’s 
attention on the goal of the 
writing task and the needs 
of the audience. 

• To Convey Experience is a 
broader representation of 
the kinds of writing students 
will be asked to do. In the 
201 1 Framework, 

“narrative” is viewed as an 
approach, not a purpose, 
and is a strategy also used 
in explanatory and 
persuasive writing. 



• The specification of 
audience on all prompts 
and at all grades will 
encourage students to 
make decisions about how 
to develop and organize 
ideas ( “approaches to 
thinking and writing”) and 
how to craft language that 
meets the needs of the 
specified audience. 



• An increase in the 

percentage of tasks for the 
persuasive purpose at 
grade 4 reflects emerging 
pedagogical practices in 
elementary schools and 
complements expectations 
for postsecondary 
preparedness at grades 8 
and 12. The distribution 
also reflects a progression 
of emphasis on writing to 
explain and to persuade, 
though many students in all 
three grades will also write 
to convey experience, real 
or imagined. 
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Exhibit 1.5 (continued). Comparison of 1998 — 2007 and 2011 NAEP Writing 

Frameworks 





1998—2007 Writing Framework 


2011 Writing Framework 


Explanation for Change 


Design of NAEP Writing Assessment 


Paper and pencil assessment for 
grades 4, 8, and 12. 


Recommendation to provide 
computer-based assessment at 
grade 4 by 2019. 

Computer-based assessment for 
grades 8 and 1 2. 


• As eighth and twelfth 
graders have become 
accustomed to composing, 
revising, and editing on 
computers, a computer- 
based assessment will 
offer students an 
environment for writing that 
more accurately reflects 
how students compose. 
Students will also have the 
option of using commonly 
available editing, 
formatting, and text 
analysis tools to compose 
their response. 

• A 201 1 computer-based 
assessment at grades 8 
and 1 2 offers students the 
opportunity to compose in 
an environment that is 
similar to that of many 
writing situations in 
postsecondary education 
and training. 




On the 1998-2007 assessment, 


The 2011 NAEP Writing 


• The 201 1 NAEP Writing 


U) 


evaluation criteria are defined as 


Assessment will evaluate three 


Assessment clearly and 




“general characteristics of writing 


broad domains of writing in all 


consistently defines criteria 




by mode”; some characteristics 


students’ responses: 


for the evaluation of 


CL 


(e.g., “organization” and 




effective writing across 
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“mechanics”) apply to all three 


• Development of Ideas 


grades. 


z 


modes, whereas others are mode- 


• Organization of Ideas 




£ V 


specific (e.g., “develops character” 


• Language Facility and Use of 


• Features of writing will be 


Evaluating Responses on 1 
Assessmei 


for the narrative mode). 


Conventions 


evaluated in relation to the 
purpose and audience 
specified in the writing 
task. 

• Development and 

organization of ideas will 
be evaluated for writers’ 
use of relevant and 
effective approaches to 
thinking and writing (e.g., 
analyzing, evaluating, 
narrating, etc.). 
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Exhibit 1.5 (continued). Comparison of 1998 — 2007 and 2011 NAEP Writing 

Frameworks 





1998—2007 Writing Framework 


2011 Writing Framework 


Explanation for Change 




On the 1998-2007 assessments, 


For the 2011 NAEP Writing 


• The Profile of Student 




student performance was reported 


Assessment, reports on student 


Writing enhances the 




in two ways: 


performance may include a new 


traditional NAEP reporting 






component. Assessment results will 


methods — scale scores 


■*-< 


• Scale scores 


be reported in three ways: 


and achievement levels — 


V > 


• Achievement Level 




by providing information 


CC 


Descriptions 


• Scale scores 


about various dimensions 


c 


• Achievement levels 


of writing and about the 


E 


Average scale scores are derived 


• Profile of Student Writing-. A 


relationship between those 


(/> 

</) 


from the overall level of 


nationally representative sample 


attributes and 


CD 

(/) 


performance of groups of students 


of student responses at each 


achievement. 


U) 

< 


on NAEP assessment items. For 


grade will be closely analyzed in 




u> 


Writing, average scale scores 


relation to the evaluative criteria 






have been expressed on a 0-300 


used to score student writing. 




k. 


scale. 


Utilizing both qualitative and 




CL 




quantitative methods, data will 




LU 


Achievement levels are 


be analyzed in order to detect 




Z 


performance standards set by 


patterns between attributes of 




o> 


NAEP that provide a context for 


the responses and performance 




Ip 


interpreting student performance. 


at the Basic, Proficient, and 




o 


These performance standards are 


Advanced levels of achievement. 




CD 


used to report what students 








should know and be able to do at 
the Basic, Proficient, and 
Advanced levels of performance 
in each subject area and at each 
grade assessed. 
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Chapter Two 



Content of the 2011 NAEP Writing Assessment 

The purpose of this chapter is to explain the content of the 2011 NAEP Writing 
Assessment. 

Introduction 



The focus of the 2011 NAEP Writing Assessment is to assess “effective communication,” 
the ability to achieve the intended aims of composing and to address the needs of an 
audience (Britton, et al., 1975). Therefore, the writing tasks on the assessment will clearly 
identify the purpose for writing and will state or clearly imply the audience on the written 
task. This approach reflects writing research on the need to develop writing tasks that 
offer students genuine opportunities to communicate (Graves, 1999; National Writing 
Project and Nagin, 2003). Moreover, the principles underlying the new writing 
framework are in accordance with the learning standards of most states (ACT Research 
Brief, 2006), which specify that effective written communication involves adjusting 
written language for specific audiences and purposes. Writing to meet specified purposes 
and audiences is also an objective of writing at the college level (Council of Writing 
Program Administrators, 1999). 

On the 2011 NAEP Writing Assessment, each student at grades 4, 8, and 12 will respond 
to two writing tasks. Topics that students will respond to on these tasks will include real- 
world, age- and grade- appropriate issues that are familiar and accessible. To increase 
students’ engagement with the topic and their motivation to write, writing tasks will be as 
open-ended as is appropriate to encourage a variety of potential responses within the 
parameters of the task. In responding to NAEP writing tasks, students may include 
information and ideas from their own reading, observations, and experiences, or they may 
be asked to respond to short reading passages or to visual stimuli such as photographs or 
simple visual displays of information. 

The 2011 NAEP Writing Assessment highlights successful writers’ negotiation between a 
task’s specified purpose and audience and the rhetorical choices writers need to make to 
communicate effectively. Writing tasks will require writers at all three grade levels to 
develop, organize, and craft the language of their text by drawing upon approaches to 
thinking and writing that are most effective for their purpose and audience. At grades 8 
and 12, the writing task may also require students to determine what form is best suited to 
their purpose for writing. Exhibit 2.1 illustrates the process involved in completing tasks 
on the 2011 NAEP Writing Assessment and the relationship between this process and the 
evaluation of the writing task. 



NAEP 201 1 Writing Framework 
19 





Exhibit 2.1. Content Components for the 2011 NAEP Writing Assessment 

Communicative Purposes 



( \ 

To Persuade 

V, / 



' > 

To Explain 

s. / 



To Convey 
Experience 




Example Audiences 



Principal 


Parents 


Peers 


Community Leaders 


Teachers 


Government Officials 


etc. 






Example Approaches to Thinking and Writing 



Analyze 

Narrate 

Reflect 



Describe 

Summarize 

Question 

etc. 



Evaluate 

Synthesize 

Argue 




Example Forms 




Essay Story Article 

Letter Editorial Report 

etc. 





Evaluation Criteria 



f N 

Development 
of Ideas 

i * 



( \ 

Organization 
of Ideas 



Language 
Facility and 
Conventions 
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Communicative Purposes for Writing 



Any piece of writing is constructed with a goal or objective in mind — a “communicative 
purpose.” While writers often try to achieve multiple goals when composing texts — such 
as affecting a reader’s emotions when trying to persuade an audience to change their 
point of view on an issue — certain overarching aims are more common to school and 
workplace settings and are also more appropriate for an on-demand writing assessment 
such as NAEP. The purposes for writing assessed by the 201 1 NAEP Writing Assessment 
are found throughout the core curriculum in K-12 education — in English language arts, 
in science, in social studies, and in mathematics — and take many forms, including short 
stories, personal narratives, essays, reports, summaries, research papers, letters, 
proposals, newspaper articles, and so on. The three communicative purposes assessed are: 

• To Persuade, in order to change the reader’s point of view or affect the reader’s 
action 

• To Explain, in order to expand the reader’s understanding 

• To Convey Experience, reed or imagined, in order to communicate individued 
and imagined experience to others 

These three communicative purposes are also important in postsecondary education and 
in the workplace, where most writing can be broadly described as persuading readers to 
change their perspectives or to take action; explaining information, issues, and ideas; and 
reflecting on experience to make thoughtful judgments and create connections between 
the writer and the audience. 

When completing writing tasks for any of these purposes, writers draw upon a variety of 
approaches to the development and organization of ideas in order to achieve their 
communicative purpose. These “approaches to thinking and writing” represent the ways 
in which all writers shape text, and include such strategies as storytelling (e.g., narrating, 
describing), logical reasoning (e.g., analyzing, synthesizing), or critiquing (e.g., 
evaluating, interpreting). These approaches are sometimes used alone, but are also often 
used in various combinations, depending on the writer’s purpose for writing. Approaches 
to thinking and writing will not be specified in the assessment writing tasks, but writers at 
all three grades will be expected to use relevant and effective approaches to developing 
and organizing their ideas. 

To Persuade 

Persuasive writing is a demanding task that requires critical thinking, complex language, 
and the ability to consider one’s own views and those of others (Nippold, Ward- 
Lonergan, and Fanning, 2005). People write to accomplish persuasive purposes 
throughout their lives: adults write to persuade in many situations, including business 
proposals, letters of recommendation, or grant applications, and persuasive writing tasks 
are found in content areas throughout K-12 education. The demands and number of 
persuasive tasks increase across the grade levels as middle school and secondary school 
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students are asked to integrate research, address many perspectives on an issue, and craft 
language to move an audience that is disinterested or holds opposing views. 

In writing to persuade, writers draw on a variety of approaches to thinking and writing 
that help them accomplish their communicative purpose. For example, someone writing 
to persuade an audience of parents with young children on the dangers of excessive 
television viewing might reflect on his or her own experiences by narrating, describing, 
and evaluating the writer’s own television viewing habits on a typical day. The writer 
might also analyze the viewing habits of Americans by comparing and contrasting 
contemporary television viewing habits with those of the past, or the writer might analyze 
the effects of television viewing on Americans’ health or social interactions. Finally, the 
writer might argue for reducing the number of hours that American children and adults 
watch television by giving reasons, based on evidence, for the positive changes in 
people’s lives that could result from a more physically active and socially involved 
lifestyle. In this example, the writer might decide to focus on extensive use of one or two 
of these approaches to thinking and writing, combine all of the approaches mentioned in 
the example, or employ others that are not included here. 

On the NAEP Writing Assessment, persuasive writing tasks will ask writers to convince 
an audience to take action or change a viewpoint on a variety of topics important to 
others (e.g., peers, school, the nation). In writing tasks for grade 4, students may be asked 
to express and support a point of view. Grade 8 students may be expected to review a 
proposal and convince others of its soundness with reasons and detailed evidence. Grade 
12 students may be assigned to compose an argument that demonstrates an understanding 
of other perspectives through its development and organization of ideas and that supports 
a position with logical reasoning. 

On the following page is an example of a writing task for the purpose To Persuade for 
grade 4. 
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Exhibit 2.2. Example Writing Task for To Persuade (Grade 4) 




Imagine that students at your school are going to select a new school mascot. A mascot is 
an animal or object used to represent a group. For example, many sports teams have 
mascots. 

Four choices are being considered as your school’s mascot: Tigers, Rising Stars, 
Dolphins, and Rockets. You have been asked to choose one of the four mascots and to 
support your choice in a letter to the school principal. 

Write a letter to your principal convincing him or her that your choice should be the 
school mascot. Be sure to include reasons and examples in your letter. 



For additional examples of To Persuade writing tasks, see appendix B. 

To Explain 

Writers who compose to explain seek to present information and ideas to others in a 
manner that aids understanding of a topic. Throughout the K-12 curriculum, writing to 
explain is the most commonly assigned communicative purpose (Graham & Perin, 2007): 
students write summaries, research reports, and other kinds of explanatory tasks in all of 
their school subjects. Writing to explain is an everyday occurrence in the workplace and 
the adult world as well, where people write informative e-mail messages, fill out 
applications, and write instructions. 

As with writing to persuade, writers seeking to explain a subject to an audience also draw 
upon a variety of approaches to thinking and writing that help them present their ideas. 
For example, a writer who wants to explain to middle school science students how DNA 
was discovered might narrate the story of the competing efforts of James D. Watson and 
Francis Crick to discover DNA. The writer might also describe the steps in the process 
that led to the discovery of DNA and analyze how each step helped scientists toward the 
discovery. To explain the importance of the discovery, the writer might evaluate the 
impact of DNA research on other scientific research and on other fields by synthesizing 
information from several perspectives. In this example, the writer might decide to focus 
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on extensive use of one or two of these approaches to thinking and writing, combine all 
of the approaches mentioned in the example, or employ others that are not included here. 

On the NAEP Writing Assessment, tasks designed to assess students’ ability to write to 
explain at grade 4 might call for a basic explanation of personal knowledge or an 
explanation of a sequence of pictures and/or steps provided in the task. Grade 8 tasks may 
ask students to analyze a process or write a response that compares similarities and 
differences between two events or ideas. Grade 12 tasks may focus on asking students to 
identify the causes of a problem or define a concept. At all three grade levels, writers will 
be expected to explain a topic to an audience by drawing upon approaches to thinking 
and writing that help them clearly present their ideas and by using language that helps the 
audience fully understand the topic or subject being explained. 

Below is an example of a writing task for the purpose To Explain for grade 12. 

Exhibit 2.3. Example Writing Task for To Explain (Grade 12) 

Members of your community, including local leaders and the mayor, are concerned about 
civic awareness and town pride. In order to open a dialogue with area residents, your 
local newspaper is inviting residents to respond to a question civic leaders have debated: 
“What makes a good community?” The newspaper wants those who respond to define a 
good community and to explain what elements are needed to create a good community. 
Responses to this question will be read by members of the city council, including the 
mayor, and used to support their efforts to improve civic awareness and town pride. 

Write a response for the newspaper in which you define a good community and explain 
what elements make a good community. Be sure to use specific examples and details to 
explain your ideas. 



For additional examples of To Explcdn writing tasks, see appendix B. 

To Convey Experience, Real or Imagined 

In conveying experience, writers seek to bring real or imagined experience to life for 
their readers. Writers connect to their audiences through descriptive details, voice, style, 
reflection on the meaning and significance of events and actions, and evocation of 
emotional response. Writing to convey experience reflects many text forms in the K-12 
and postsecondary curricula: stories, personal statements and reflective essays, 
biographies, satirical essays, and so on. Writing to convey experience is commonly used 
throughout life to establish and convey identity, in forms such as blogs, interviews, and 
memoirs. 

Narration is a common approach used in writing to convey experience, but writers also 
draw upon other thinking and writing approaches. For example, in writing to convey a 
particularly important event in one’s life to peers, a writer might analyze the causes of the 
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event, and then tell the story of the event by narrating what happened and describing the 
setting and others who were involved. The writer might also reflect on the meaning of the 
event by sharing thoughts about how it has affected his or her attitudes or actions. The 
writer might also evaluate the importance of the changes brought about by the event. In 
this example, the writer might decide to focus on extensive use of one or two of these 
approaches to thinking and writing, combine all of the approaches mentioned in the 
example, or employ others that are not included here. 

Tasks designed to assess writing to convey experience on the 201 1 NAEP Writing 
Assessment will encourage flexibility in thinking and form, allowing writers to organize 
complex reflective or imaginative responses. A grade 4 student could be asked to craft a 
short story by narrating events and describing setting and characters. A grade 8 student 
could be asked to write a narrative account of a favorite memory or experience and 
convey how it affected the writer and others. A grade 12 student might use writing to 
explore, reflect, and evaluate aspects of his or her character or growth as a person, using 
extended examples and self-questioning to illustrate the discussion. 

Below is an example of a writing task for the purpose To Convey Experience for grade 8. 

Exhibit 2.4. Example Writing Task for To Convey Experience (Grade 8) 



Teen Life, a magazine for young adults, has announced a writing contest for middle 
school students. The theme of the contest is “Achieving Gocds. ” The magazine has 
published the following contest instructions: 

Teen Life wants to hear about the experiences of young people who have achieved goals. To 
enter, write about a memorable moment in your life when you achieved a goal you set for 
yourself. All successful responses will need to clearly convey the experience of achieving a goal 
so that the reader can fully understand the experience and its importance. The staff of Teen Life 
will select a winner, which will be published in next month’s issue. 

Write a response for the contest, describing an experience of achieving a gocd and the 
importance of that experience to your life. Be sure to include details in your response that 
help readers understand your experience and its importance. 



For additional examples of To Convey Experience writing tasks, see appendix B. 

Audience 



The intended recipient of a written text plays an important role in shaping the writer’s 
approach to all dimensions of his or her response. From developing and organizing ideas 
to the language the student uses, clear and effective communication requires an 
awareness of readers’ needs and their level of knowledge about the writing topic. For this 
reason, audience will be clearly specified or implied in all writing tasks on the 2011 
NAEP Writing Assessment. 
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The kinds of audiences specified on the 2011 NAEP Writing Assessment will differ at 
grades 4, 8, and 12 based on grade-level expectations and students’ familiarity with 
writing for other audiences. These audiences will progressively broaden: grade 4 students 
will be asked to write for peers or familiar individuals, while grade 12 students may be 
asked to write for more distant, authoritative audiences, such as government officials or 
community leaders. 

Student Choice of Form 



The term “form” refers to the organizational features customarily required by a particular 
kind of writing, such as a short story, friendly letter, letter of complaint, and so on. Since 
a writer’s purpose and audience should determine the selection of the most appropriate 
form, several forms may be effective for a given writing task. On the 2011 NAEP 
Writing Assessment, students at grades 8 and 12 may have the flexibility to choose the 
form they believe best accomplishes the assigned communicative purpose for the 
audience the task specifies. For example, in response to a task asking students to “explain 
what the concept of community means to you, ” a student in grade 12 might choose to 
address the task by writing a definition essay, an editorial/opinion column, a letter to the 
City Council, and so on. In this instance, the writer would choose the form that, in his or 
her judgment, is most effective for communicating ideas to the intended audience. 

Framework developers, however, recognize that it is necessary to make writing tasks 
accessible within the time allotted for writing, so it may be necessary for some or all 
tasks to explicitly state which form a writer will use. Therefore, the 201 1 NAEP Writing 
Framework recommends that three types of writing tasks be considered at grades 8 and 
12 : 

• Tasks that specify a text type, or form (e.g., “Write a letter to persuade the 
principal of your school to . . . .”) 

• Tasks that do not specify a text type (e.g., “Persuade the principal of your school 
to....”) 

• Tasks that recommend several possible text types (e.g., “Write a letter, editorial, 
or essay to persuade the principal of your school to . . . .”) 

Given grade 4 students’ limited experience writing without a specified form, the NAEP 
writing tasks at grade 4 will clearly state the form the response should take. 

Example Task Illustrating Communicative Purpose, 

Audience, Example Approaches to Thinking and Writing, and 
Potential Forms 



The negotiation between the parameters of the written task and the choices the writer 
makes within these parameters is central to the 201 1 NAEP Writing Assessment. Writing 
tasks will specify the topic, the purpose for writing, the audience, and (at grade 4) the 
form, but writers will be expected to craft language and draw upon approaches to 
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thinking and writing that they feel will best achieve their communicative purpose. 
Students at grades 8 and 12 may be asked to choose the form that they believe best 
supports the communicative purpose of the writing task. 

In the example task on the following page, all content components of the assessment are 
identified: the writing task, the communicative purpose, the audience, some examples of 
approaches to thinking and writing that a writer might draw upon, and possible text 
forms. 
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Exhibit 2.5. Example Task Illustrating Communicative Purpose, Audience, Example 
Approaches to Thinking and Writing, and Potential Forms 



Example Task 

Scientists recently announced that a state 
park in your state contains large amounts of 
oil — a substance that can be converted into 
gasoline for cars or used to heat homes. 
Your state legislature is now debating 
whether to allow energy companies access 
to the protected land in this park in order to 
drill for the oil. In a speech to residents of 
the state, one of the state legislators 
identifies two sides of the debate: 

“The debate over whether the state should 
allow energy companies to drill for 
petroleum on protected land is important to 
this state’ s future. Some in the state 
legislature believe that allowing access to 
lands currently protected would benefit our 
economy, creating thousands of jobs and 
lowering the costs of goods and sendees. 
Others, however, believe it is more 
important to protect our natural 
environment and support a high quality of 
life for citizens of this state. 

The state legislature has called on citizens 
of the state to express their opinions about 
this issue. Take a position and write a 
response persuading members of your state 
legislature to support your position on 
whether or not protected land in your state 
should be opened to energy companies for 
drilling. 



Analysis of Task 

Grade: 12 

Purpose: To Persuade (write a response 
persuading members of your state 
legislature to support your position on 
whether or not protected land in your state 
should be opened to energy companies for 
drilling) 

Audience: Members of the student’s state 
legislature 

Examples of Approaches to Thinking 
and Writing: Students might address this 
writing task using multiple approaches, 
alone or in combination with each other. 
For example, students could: 

• Discuss the role of governments in 
protecting natural resources and whether 
economically beneficial land should be 
preserved ( analyze , evaluate). 

• Summarize and review current attitudes 
toward the issue, noting how these 
attitudes have affected the environment or 
economy ( summarize , analyze). 

• Critique the solutions offered by 
proponents and opponents of drilling, 
conceptualizing an alternative by finding a 
central idea that highlights opposing 
viewpoints ( evaluate , synthesize). 

• Analyze the importance of state parks or 
strong industries to the state ( analyze , 
evaluate). 

• Describe what an area in the state might 
look like if the land is or is not open to 
energy companies to drill oil (describe). 

• Write personal anecdotes illustrating how 
their experiences with state parks or 
protected natural areas inform their views 
on the issue ( describe , narrate, and 
reflect). 

Potential Forms: Letter, essay, opinion 
piece 
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Chapter 3 



Design of the 2011 NAEP Writing Assessment 

The purpose of this chapter is to explain the design of the 2011 NAEP Writing 
Assessment. 

Writing with Computers on the 2011 NAEP Writing Assessment 



In 2011 and beyond, the NAEP Writing Assessment at grades 8 and 12 will measure 
computer-based writing. “Computer-based writing” means that students compose and 
construct their responses using word processing software, with commonly available word 
processing tools enabled. The 2011 NAEP Writing Assessment will measure how well 
students write when using word processing software on a computer. 

The move to a computer-based assessment from traditional paper and pencil assessments 
acknowledges the vital role computers play in both student composition and writing 
instruction. Word processing programs are a common composing mode for multi-draft 
process writing in middle school language arts classes (Christy, 2005), and are routinely 
used by high school students to create complex texts or to collaborate during the writing 
process (Thomas, 2005; Perry, 2005). In fact, two separate studies conducted in 2002 
found that 90 to 94 percent of middle and high school students reported using computers 
to produce drafts during the school year; 86 percent of teachers surveyed in one study 
said they used word processing for in-class writing instruction that year (Goldberg, et al., 
2003; Horkay, et al., 2006). Results from two separate series of student background 
questions on the 2002 NAEP Writing Assessment also found that more than 75 percent of 
grade 8 and 12 students always or occasionally used computers throughout the writing 
process, while more than 70 percent of grade 12 students used word processing tools to 
compose and revise their drafts (Horkay, et al., 2006; Applebee & Langer, 2006). 

As writing on computers becomes an increasingly common means of composing, a 
computer-based assessment is the best method of understanding the writing practices of 
K-12 students. The 2011 NAEP Writing Assessment reflects students’ preference for the 
composing mode that more easily and efficiently allows them to generate and revise texts 
(Russell & Plati, 2000), and research has shown that students prefer to compose on 
computers when completing a large-scale writing assessment (McClarty, Keng, and 
Davis, 2006; Bennett, 2002). A computer-based writing assessment also provides 
students with writing environments more similar to those of postsecondary education and 
the workplace than a paper and pencil assessment can provide. 

Writing with Commonly Available Word Processing Tools 

The 2011 NAEP Writing Assessment measures computer-based writing skills: students 
will complete writing tasks using word processing software and the software’s commonly 
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available tools. In the context of NAEP, “tools” refers to the extensions built into word 
processing software that help writers modify or revise their text documents. These 
tools — such as paragraph formatting; copy, cut, and paste, or thesaurus — are often used 
by students when producing texts. Tools help students revise and rethink their writing 
during the process of composing (Cramer & Smith, 2002) and have not been shown to 
have a detrimental effect on the development of language skills (Grejda & Hannafin, 
1992). In fact, the availability of a fuller set of tools on a computer-based writing 
assessment has been shown to lead to considerable improvement in the development of 
ideas in students’ responses to an assessment task (Russell, et al., 2004). 

The following list includes tools that are commonly available to writers now and gives 
examples of the kinds of tools that will be considered for the 2011 NAEP Writing 
Assessment, pending the nature of tools available in 2011 and beyond. 

Prewriting/Planning Tools : those used to help the student prepare to compose, 
such as scratch paper to brainstorm or a simple text editor (such as Notepad) to 
outline or take notes. 

Editing Tools : those used to add, delete, correct, or adapt typed text, such as copy, 
cut, paste, undo/redo, clipboard, and select all. 

Formatting Tools : those used to alter the physical appearance of the text, 
including the fonts, paragraphs, margins, and general layout. Current examples of 
such tools include paragraph formatting and templates. 

Spelling Tools: those used to verify or suggest options for the accurate spelling of 
typed words or phrases. 

Grammar Tools: those used to verify or suggest options for altering the grammar, 
usage, and mechanics of typed words, phrases, and sentences. 

Reference Tools: those used to verify the meaning or usage of a word, or to find a 
synonym/antonym for a word or phrase. These tools include electronic 
dictionaries and thesauruses. 

Composing tools considered irrelevant to an assessment of writing or distracting to 
students — including such tools as clip art, font color, and the Internet — will not be 
enabled on the 2011 NAEP Writing Assessment. 

A computer-based writing assessment creates additional opportunities to develop 
assessments that align with what students know and can do but requires considerable 
resources and cooperation to be administered successfully. Extensive research, study, and 
field testing will be necessary before implementing the components of a computerized 
NAEP Writing Assessment. Considerations related to the delivery and administration of a 
computer-based assessment, security, collection of responses, and reporting results will 
be considered in detail in the 2011 NAEP Writing Assessment Specifications. 
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Computer-Based Testing at Grade 4 

A computer-based assessment at grade 4 has been impractical because of limited 
instructional time for computer use prior to grade 4 and the limited keyboarding 
proficiency of many grade 4 students. However, the grade 4 assessment should become 
computer-based at some point during the tenure of the 2011 NAEP Writing Framework, 
potentially by 2019. By this time, computer instruction and keyboarding proficiency 
among grade 4 students should support a computer-based assessment. Pursuant to the 
Governing Board’s May 2010 decision, NAEP is planning a computer-based writing 
assessment at grade 4 in 2013. 

Time per Task 



For the 2011 NAEP Writing Assessment, students will be presented with two extended 
response tasks. This design is the best means of constructing a fair and effective large- 
scale survey of student achievement in writing. The 2011 NAEP Writing Assessment will 
be administered as two 30-minute writing tasks. 

Distribution of Communicative Purposes 



Exhibit 3.1 shows the distribution of tasks at each grade level in relation to 
communicative purposes on the 2011 NAEP Writing Assessment. The percentages 
represent the proportion of tasks designed to assess each purpose out of the total number 
of tasks at each grade. 



Exhibit 3.1. Percentage Distribution of Communicative Purposes by Grade 



Purpose 


Grade 4 


Grade 8 


Grade 12 


To Persuade 


30% 


35% 


40% 


To Explain 


35% 


35% 


40% 


To Convey Experience 


35% 


30% 


20% 



The assessment measures responses to a variety of communicative purposes at all grades, 
with an increase of To Persuade tasks at grades 4, 8, and 12, a consistent range of To 
Explain tasks among all three grades, and a decrease in the distribution of To Convey 
Experience tasks across the later grades. Students are accustomed to writing for all three 
communicative purposes in grades and subject areas throughout the K-12 curriculum, 
and this distribution highlights the particular importance of To Convey Experience and To 
Explain tasks in elementary and middle school, and To Persuade and To Explain tasks in 
secondary school settings. This progression also reflects the increasing importance of 
these two purposes in postsecondary and workplace settings. 

Assessing Students with Special Needs 



The framework committees have endeavored to design a writing assessment that allows 
for the participation of the widest possible range of students at grades 4, 8, and 12. 
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Students with a range of backgrounds and experiences are included in the assessment, 
including English language learners (ELL) and students with disabilities (SD) who, based 
on inclusion criteria provided by NAEP, are capable of participating. 

Item and assessment development procedures will be followed that build accessibility 
into the assessment, while assessing student ability to write in English. Tasks on the 2011 
NAEP Writing Assessment will be as fully and clearly explained in order to allow more 
sophisticated tasks to be included without disadvantaging students who have limited 
English language proficiency. Writing tasks that rely heavily on cultural or experiential 
knowledge for a successful response will not be included in the assessment. 

Special care will be taken to ensure that all student populations have an equal opportunity 
to demonstrate what they know and are able to do on the 2011 NAEP Writing 
Assessment. As many students as possible should be encouraged to participate in the 
NAEP Writing Assessment. Accommodations should be offered, if necessary, to enable 
students with disabilities and English language learners to participate, but should not alter 
the knowledge and skills being assessed as defined in this framework. 

Inclusion Criteria for Students with Disabilities 

A student identified as having a disability — that is, a student with an Individualized 
Education Program (IEP), a Section 504 Plan, or an equivalent classification — will be 
included in the NAEP assessment unless the student has previously been identified in an 
Individualized Education Program (IEP) as having the most significant cognitive 
disabilities and is assessed by the state on an alternate assessment based on alternate 
achievement standards. 

In deciding how a student will participate in NAEP: 

a) If the student has an Individualized Education Program (IEP) or Section 504 plan 
and is tested without accommodation, then he or she takes NAEP without 
accommodation. 

b) If the student’s IEP or 504 plan specifies an accommodation permitted by NAEP, 
then the student takes NAEP with that accommodation. 

c) If the student’s IEP or 504 plan specifies an accommodation or modification that 
is not allowed on NAEP, then the student is encouraged to take NAEP without 
that accommodation or modification. 

(National Assessment Governing Board, 2010) 

Inclusion Criteria for English Language Learners 

The NAEP program has established procedures to include and accommodate as many 
English language learners as possible in NAEP assessments. All English language 
learners selected for the NAEP sample who have been in United States schools for one 
year or more should be included in the National Assessment. One year or more shall be 
defined as one full academic year before the year of the assessment. School staff make 
the decisions about whether to include ELL students in the NAEP assessments, and 
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which testing accommodations, if any, they should receive. The NAEP program furnishes 
tools to assist school personnel in making those decisions. 

A sample of students is first selected at each grade level being tested. Students are 
selected for participation in the NAEP assessments without regard to English language 
learner (or disability) status. Accommodations should be offered that maximize 
meaningful participation, are responsive to the student’s level of English proficiency, and 
maintain the knowledge and skills being assessed as defined in this framework. 

A student may still be able to participate in the assessment even if the student did not 
participate in the regular state assessment, or even if he/she generally uses 
accommodations NAEP does not allow. In either instance, school staff would be asked 
whether that student could participate in NAEP with the allowable accommodations. 

Accommoda tions 

Some students may need accommodations to be able to participate in the 2011 NAEP 
Writing Assessment. Accommodations in NAEP assessments include: 

• Extended time in regular session 

• Small group testing 

• One-on-one testing 

• Large-print booklets 

• Other: format or equipment accommodations such as a Braille version of 
the assessment, amplification devices, or magnification equipment (if 
provided by the school) 

Accommodations will be offered in combination as needed (e.g., students who receive 
one-on-one testing generally also use extended time) and attempts will be made to match 
the accommodations students are allowed in school and on their state assessment(s). To 
ensure consistency in administering the assessment, accommodations will be 
standardized to as great an extent as possible across the 2011 NAEP Writing Assessment. 
Appropriate accommodations for English language learners and students with disabilities 
will also be available for computer-based delivery of the assessment. 

Additional information about assessing students with special needs is included in the 
2011 NAEP Writing Specifications. Detailed manuals and training procedures will also 
be developed by NAEP contractors to ensure that inclusion criteria are applied uniformly 
and that students are assessed appropriately with accommodations. 
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Chapter 4 



Evaluation of Responses on the 2011 NAEP Writing 

Assessment 

The purpose of this chapter is to explain how students’ responses will be evaluated for 
the 2011 NAEP Writing Assessment. 

Which Features of Writing Will Be Evaluated 



The most successful writers consistently make effective choices in all dimensions of their 
writing to achieve the writing task’s purpose and to meet the needs of their intended 
audience. The 2011 NAEP Writing Assessment has been constructed to emphasize that 
many important features of writing contribute to the overall quality of a written response. 

For the 2011 NAEP Writing Assessment, three broad domains will be evaluated at grades 
4, 8, and 12: 

• Development of Ideas 

• Organization of Ideas 

• Language Facility and Conventions 

These broad domains are consistent with learning standards and benchmarks in most 
states. In 2006, over 90 percent of states’ standards, benchmarks, and assessments at 
grades 4, 8, and 12 included development and organization of ideas, style and varied 
sentence structure, vocabulary and precise diction, and conventions of standard written 
English (grammar, usage, punctuation, capitalization, and spelling). In addition, 
approximately 60 percent of states had benchmarks for complexity of ideas and the 
logical progression of ideas, primarily at grades 8 and 12 (ACT Research Brief, 2006). 

Representatives from higher education and workplace settings also agree that a strong 
demonstration of many important features of writing is critical for success in these 
postsecondary arenas. Postsecondary educators value abstract reasoning and marshalling 
strong evidence in support of an argument, while representatives from various workplace 
settings tend to emphasize the importance of clarity and conciseness. All of these groups, 
however, emphasize the importance of awareness of audience, strong organization of 
ideas, and correct use of the conventions of standard written English (ACT Research 
Brief, 2006). 

Exhibit 4.1 presents the three broad domains assessed by the 2011 NAEP Writing 
Assessment and the features within each domain that will be used to evaluate student 
performance on the writing tasks. 
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Exhibit 4.1. Criteria for Evaluating Responses on the 2011 NAEP Writing 

Assessment 



Development of ideas is effective in relation to the writer’s purpose and audience. 

• The depth and complexity of ideas are effective in relation to the writer’ s purpose 
and audience. 

• Approaches to thinking and writing (e.g., analyzing, synthesizing) are used 
effectively in relation to the writer’s purpose and audience. 

• The details and examples used to develop ideas are specific and effective in relation 
to the writer’s purpose and audience. 

Organization is logical in relation to the writer’s purpose and audience. 

• Text structure is logical and effective in relation to the writer’s purpose and to the 
approaches to thinking and writing that the writer has used. 

• Coherence is maintained within and between paragraphs. 

• Focus is maintained throughout the response. 

Language facility and conventions support clarity of expression and the effectiveness of 
the writing in relation to the writer’s purpose and audience. 

• Sentence structure is well controlled and sentence variety is appropriate for the 
writer’ s puipose and audience. 

• Precise and appropriate word choice supports clarity of expression and enhances the 
presentation of the writer’s ideas. 

• Voice and tone are effective in relation to the writer’s purpose and audience. 

• Grammar, usage, and mechanics (capitalization, punctuation, and spelling) support 
clarity of expression and enhance the presentation of the writer’s ideas. 



Development of Ideas 

To communicate thought to others, writers need to develop ideas to go beyond the 
presentation of superficial assertions — or mere opinions — which are likely to raise 
unanswered questions about the writers’ thoughts or experiences. When informing or 
persuading the audience, for example, writers must show in detail how generalities apply 
to particular cases, the basis for their conclusions, or how their conclusions differ from 
those of others. When conveying experience, writers often need to include extensive and 
exact descriptive detail to communicate a clear understanding of the experience. 

Although not all forms of writing require the same kind or extent of development (e.g., a 
detailed analysis of a problem versus a brief business letter or memorandum), effective 
writing is dependent on writers’ ability to anticipate the questions that might arise in the 
reader’s mind. Effective writers judiciously address those questions, with the depth and 
extent of the development of ideas determined by the difficulty of the material to be 
explained and by the purpose and audience for the writing. 
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As Exhibit 4. 1 illustrates, the following features will be used to evaluate the 
Development of Ideas in a written response. 

Depth and Complexity 

Successful writers show depth and complexity by demonstrating insight, knowledge, 
and/or understanding that allows them to move beyond a superficial discussion of a topic 
and establish credibility with their readers. Some of the most common means of creating 
substance include examining the relationships between a topic and related concepts, 
examining the topic from a global or “big picture” perspective, and/or demonstrating a 
broad understanding of how a topic might be perceived by the writer’s audience or by 
other readers. On To Persuade tasks, then, successful writers might demonstrate depth 
and complexity by examining different perspectives on an issue and by providing 
convincing evidence to support the writer’s own position. On To Explain tasks, success- 
ful writers might demonstrate depth and complexity by providing a thoughtful and 
insightful explanation of the subject in terms the reader can readily understand or by fully 
examining the relationship of its parts. 

Approaches to Thinking and Writing 

The 2011 NAEP Writing Framework emphasizes that a number of approaches to thinking 
and writing (e.g., describing, evaluating, reflecting/questioning) may be used effectively 
to support the development of ideas and connect with the reader. Successful writers draw 
upon relevant approaches to thinking and writing that enhance their communicative 
purpose, or they extend the development of their ideas by weaving multiple approaches 
into a successful written response. Specific approaches to thinking and writing will not be 
specified on NAEP tasks, but responses will be evaluated for their use of effective 
approaches in relation to the development and organization of ideas. 

Details and Examples 

Successful writers develop responses by providing compelling details and examples that 
illustrate the writer’s points and expand the reader’s understanding. For example, on a 
task asking students to explain to a curious new student what to expect on the student’s 
first day of class, a writer might discuss recess and lunchtime rules, describe the way the 
cafeteria smells at lunchtime, and/or tell a story illustrating how kind the teachers are to 
the students. For the 201 1 NAEP Writing Assessment, students will be able to develop 
their ideas with details and examples taken from their own experiences or observations, 
or, in some cases, from brief reading passages or visual stimuli. 

Organization of Ideas 

Organization refers to the logical arrangement of sentences and paragraphs in order to 
coherently express ideas that readers can understand and follow. Organization is thus a 
fundamental component of effective writing — no matter the purpose, the form, or the 
audience — because the order and presentation of ideas is an important component of what 
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compels readers to be convinced, enlightened, or affected. If writing is poorly organized, 
the writer’s ideas will not be clearly conveyed and readers are likely to become confused 
or frustrated. Good organization requires the writer to remain focused on the writing topic 
by establishing a clear presentation and progression of ideas. 

The following text features will be used to evaluate Organization of Ideas. 

Logical Text Structure 

The rhetorical flexibility available to the writer on the 201 1 NAEP Writing Assessment 
will impact how the written response is constructed and how ideas are arranged. Students 
may respond to the topic, purpose, and/or audience in any number of different ways to 
best accomplish the demands of the writing task, leading to many kinds of effective text 
structures. For example, the use of analysis to explain might result in comparing and 
contrasting two or more ideas or objects, or an evaluation intended to persuade the reader 
might address potential solutions to a problem in some order of priority. In a text written 
To Convey Experience , writers might use a basic chronological approach to telling a 
story, or they might use other narrative techniques (e.g., flashbacks). Writers will be 
evaluated for the way their approach to organization helps build a logical progression of 
ideas to achieve the purpose of the task and to communicate to an audience. 

Eighth and twelfth graders will also draw upon their knowledge of form to structure their 
texts. Successful writers will use a form and structure appropriate to the topic, purpose, 
and audience when responding to a task on the assessment. For example, when asked to 
persuade members of a state legislature to support a particular position about drilling for 
oil in state parks, a student might effectively structure her ideas in the form of a news- 
paper editorial addressing and refuting points of opposition to her beliefs. Depending on 
the purpose and audience for the task, other potential forms eighth and twelfth graders 
may use include letters, different kinds of essays, articles, reports, and so on. 

Coherence and Focus 

Successful writers maintain focus by ensuring that ideas, details, and examples are 
relevant to the purpose, topic, and audience for the text. A written text is coherent if its 
ideas are clearly connected within and between paragraphs. Similarly, a text is coherent if 
the sentences within paragraphs and the paragraphs themselves are presented in a clear 
and logical order. Strong use of transitions helps to ensure that the reader perceives the 
relationship among parts of the writing and the relationship of the parts to the whole. 

Language Facility and Conventions 

Language facility refers to stylistic effectiveness and grammatical clarity in the ways 
writers express ideas to the reader. Good writers make many choices about language use. 
They decide what kinds of sentences to use and how to construct sentences to clearly 
convey relationships among ideas; they also purposefully arrange these sentences to 
enhance the organizational qualities of their writing. They choose particular words and 
alter how emotion and voice are expressed to clearly and effectively communicate 
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meaning — and to maintain the reader’s interest. They adhere to established rules of 
communication to ensure understanding and avoid distractions. 

The following text features will be used to evaluate students’ language facility and 
conventions. 

Sentence Structure and Sentence Variety 

Good writers craft the structure and variety of their sentences to illuminate their topic, to 
effectively accomplish the purpose of the task, and to engage the reader. Effective 
sentence structure can also enhance the development and organization of ideas and is a 
device used by writers to emphasize ideas within a sentence. For example, students on the 
2011 NAEP Writing Assessment might use parallel sentence structure to demonstrate the 
comparability of points when explaining similarities or differences, or they might 
combine clauses with colons or semicolons in order to persuade an audience that their 
ideas are connected and logical — and thereby worth believing. 

Alternating the length and kinds of sentences used can also make a text easier to follow 
and more interesting to read. Successful writers achieve sentence variety by strategically 
crafting shorter and longer sentences, by varying the ways sentences begin, and by using 
a variety of sentence types (e.g., simple, compound, complex). These stylistic choices are 
made in order to best express or develop ideas that help the writer achieve the 
communicative purpose of the task. 

Word Choice, Voice, and Tone 

Successful writers are able to decide what words will most precisely and clearly express 
their ideas, and what words will support their purpose for writing. Successful writers also 
consider their relationship with the reader, choosing words that encourage a reader to 
connect to their ideas and intentions. When conveying experience, for example, writers 
often choose evocative words that appeal to the reader’s senses and give the sensation of 
experiencing the event. 

Effective writing also involves adapting word choice, voice, and tone depending on the 
purpose, audience, and/or topic of a writing task. In other words, the most successful 
writers recognize the context of the writing situation — what they are writing about, who 
their audience is — and deliberately alter their style and language to achieve a purpose. 
Two ways this is achieved are through voice, the writer’s ability to convey a personality 
or attitude in language, and tone, the writer’s attitude toward the topic or audience. 

Writers alter their manner of expression (e.g., their word choice, sentence structures, etc.) 
as a means of demonstrating their attitudes towards a topic or an audience. For example, 
a writer composing a satirical essay may express a tone of mockery or disgust for a topic 
by altering his or her voice to include common elements of sarcasm such as exaggeration 
or rhetorical questions. The 2011 NAEP Writing Assessment will evaluate how these 
stylistic choices help the writer achieve the purpose of the writing task and communicate 
effectively to the audience. 
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Conventions: Grammar, Usage, and Mechanics 

Grammar is the system through which a given language is ordered according to an 
agreed-upon set of internal rules; usage refers to established conventions of written 
language commonly used in forms of communication; and mechanics refers to conven- 
tions of capitalization, punctuation, and spelling. Good writers have a command of 
grammar, usage, and mechanics so that only minimal errors, if any, are present in their 
writing. Just as the variety, severity, and pervasiveness of errors in grammar, usage, and 
mechanics can distract readers and make it difficult to understand the writer’s meaning, 
correct use of language can facilitate understanding by allowing the reader to focus on the 
writer’s thoughts and ideas. 

How Responses Will Be Evaluated 



Scoring Rubric for Each Communicative Purpose 

The use of a holistic rubric is recommended as the basic tool for evaluation of responses 
on the 2011 NAEP Writing Assessment. The holistic approach to scoring focuses on an 
evaluation of the whole response rather than on its individual parts (Myers, 1980). That 
is, a response will not be evaluated with a separate score for each writing feature, and an 
overall score will not be derived by adding together scores for each separate feature. 
Instead, a response will be scored by assessing performance across multiple criteria — 
development of ideas, organization of ideas, language facility and conventions — to 
evaluate overall performance. Because of the depth of evaluative criteria and the wide 
range of potential responses to the writing tasks, responses to tasks on the 2011 NAEP 
Writing Assessment will be scored on a six point scale, with 1 being low and 6 being 
high. A six -point scale provides the clearest possible distinction among achievement 
levels, the most explicit operational definitions, and a continuum of performance 
(Wolcott, 1998). 

To score responses for the writing assessment, a holistic rubric unique to each communi- 
cative purpose will be used. This approach will result in the use of three rubrics: one for 
the communicative purpose To Explain , another for the purpose To Persuade, and a third 
for the purpose To Convey Experience. Each scoring rubric will contain all the features to 
be evaluated and will include descriptions of the performance expected at each score 
point. 

Although the same scoring rubric will be used for each communicative purpose across 
grades 4, 8, and 12, the interpretation, or application, of the rubric will be different at 
each grade. Scorers will be carefully and extensively trained to interpret and apply the 
scoring rubric by reading actual responses selected at each grade level. These responses 
will be chosen to demonstrate appropriate and increasing expectations for performance at 
each grade level. For example, responses selected to guide evaluation of writing at grade 
4 for the purpose To Persuade might, at the highest level of performance, use some 
specific and relevant details to support the writer’s opinion. At grade 12, however, 
responses selected to demonstrate performance at the highest level might demonstrate the 
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ability to consistently present reasons and details that show an awareness of other 
perspectives on the issue. 



Preliminary Scoring Rubrics 

Final scoring rubrics will be developed by using actual student responses obtained during 
field testing of 201 1 NAEP Writing Assessment tasks. All features of writing to be 
evaluated will be present in the final scoring rubrics for the 201 1 NAEP Writing Assess- 
ment. That is, the features used as a basis for evaluating responses will not change. 
However, the language used to describe expectations for the level of performance may 
differ in the final scoring rubrics based on actual performance found in the field test 
responses. 

The preliminary scoring rubric for To Persuade uses the following description of the 
qualities of a response at score point 5. The words and phrases describing the level of 
expectation at this score point are italicized in the excerpt below. These are the kinds of 
descriptors that may be revised based on actual performance in field test responses prior 
to scoring actual responses from the 2011 NAEP Writing Assessment. 

Score = 5 Responses in this range demonstrate competent skill in responding to the writing 
task. Elements are usually well controlled and clearly support the writer’s purpose, 
audience, and form. 

• The response formulates a position that usually recognizes and acknowledges multiple 
significant aspects of the issue but may not fully address some of the complexities of the 
issue. The response demonstrates some insight by acknowledging and partially 
addressing other perspectives, by evaluating some implications of the writer’s position, 
and/or by using affective arguments that are usually persuasive. 

• The response usually provides persuasive reasons and evidence to support the writer’s 
position. Approaches to the development of ideas are usually used skillfully to support 
the persuasive purpose. 

• Ideas are usually focused on the topic. Organization is clear and may demonstrate a 
logical progression of ideas that supports the persuasive purpose and is relevant to the 
writer’s approaches to organization. Transitions clearly convey relationships among 
ideas. 

• Sentence structure is well controlled to communicate relationships among ideas and 
varied as appropriate for the writer’s purpose. Word choice is usually precise and 
evaluative and usually supports the persuasive purpose. Voice and tone are usually 
controlled and effective for the writer’ s purpose and audience. 

• Grammar, usage, and mechanics are usually correct with a few distracting errors, but 
meaning is clear. 

Preliminary holistic scoring rubrics for each of the three communicative purposes are 
provided in appendix C. 
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Training Readers to Score Responses 



In a large-scale assessment like NAEP, scorers are taught to use the scoring rubric 
through extensive training using many example responses at each score point. Scorers are 
also required to demonstrate their ability to score accurately by passing a qualification 
test. Responses to writing tasks on the 2011 NAEP Writing Assessment will likely vary 
widely, so scorers will be carefully trained to evaluate the effectiveness of responses that 
may look very different from one another. Therefore, the responses used to train scorers 
will be selected to demonstrate many different examples of how writers employ different 
approaches when communicating ideas to the audience identified in the writing task. 

In addition, scorers will be trained to evaluate students’ responses as on-demand writing. 
Scorers will be trained to assess responses based on all of the features included in the 
scoring rubric, but they will take into consideration that the time limit for writing does 
not allow writers to pursue all the possibilities for revision and editing of their initial 
draft. 
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Chapter 5 



Reporting Results of the 2011 NAEP Writing Assessment 

The purpose of this chapter is to explain how the results of the 2011 NAEP Writing 
Assessment will be reported. 

How NAEP Results are Reported 



The National Assessment of Educational Progress provides the only national report on 
student achievement in a variety of subjects. NAEP administers writing assessments at 
regular intervals to students in grades 4, 8, and 12 attending both public and nonpublic 
schools, collecting a significant, representative sample of student writing at these grades. 
The 2007 NAEP Writing Assessment, for example, sampled 20,000 grade 12 students 
from public and private schools at the national level. In 2007, NAEP also sampled 
152,000 grade 8 students at the state and national levels from public and private schools 
and from the districts participating in NAEP’s Trial Urban District Assessment. 

The primary means for public release of NAEP results is a printed summary report 
known as The Nation’s Report Card. This report is also available on a dedicated website: 
http://nationsreportcard.gov. Both resources provide detailed information on the nature of 
the assessment, the students who participate, and the assessment results. 

The Nation’s Report Card includes information on the performance of various subgroups 
of students at the national, state, and urban district levels. Subgroups for NAEP include: 

• Gender 

• Race/Ethnicity (White, Black, Hispanic, Asian/Pacific Islander, American 
Indian/ Alaska Native) 

• Eligibility for Free/Reduced-Price Lunch 

• Students with Disabilities 

• English Language Learners 

Detailed data on NAEP results, demographic variables, and subject-specific background 
information are available via the NAEP Data Explorer. Additional restricted data are 
available for scholarly research, subject to National Center for Education Statistics 
(NCES) licensing procedures. 

The Nation’s Report Card also reports performance for public schools in states and 
jurisdictions and the NAEP Trial Urban Districts. The Trial Urban District Assessment 
(TUDA) was initiated in 2002 in five large urban school districts that are members of the 
Council of the Great City Schools (Atlanta, Chicago, Houston, Los Angeles, and New 
York). Five additional districts were added in 2003 and 2005: Boston, Charlotte - 
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Mecklenburg, Cleveland, San Diego, and Austin. These ten districts participated in the 
2007 NAEP Writing Trial Urban District Assessment. 

In addition to The Nation’s Report Card, the NAEP website provides access to other 
NAEP reports, sample writing tasks, and scoring rubrics with example responses. The 
website also lists data sources that can be used for more in-depth analysis of student 
achievement results or of the assessments themselves. The Nation’s Report Card and its 
website include references and links to the National Assessment Governing Board’s 
website and to the National Center for Education Statistics website. 

Reporting Scale Scores and Achievement Levels 



NAEP writing results are reported in two ways: as scale scores and as percentages of 
students attaining achievement levels. Scale scores, which are derived from student 
responses to NAEP assessment items, summarize the overall level of performance 
attained by a group of students. In 1998, 2002, and 2007, NAEP writing scale scores 
were presented as average scale scores on a 0-300 scale and as scale scores at selected 
percentiles. For example, grade 8 students who scored 180 on the 2002 administration 
fell into the 75th percentile. When used in conjunction with interpretive aids, such as item 
maps, scale scores provide information about how much students know and can do in 
writing. Performance for each grade is scaled separately; therefore, average scale scores 
cannot be compared across grades. For example, equal scores on the grade 4 and grade 8 
scales do not imply equal levels of writing ability. The scale score range for 2011 will be 
determined during the assessment development process. 

Reporting on achievement levels is the primary way in which NAEP results reach the 
general public and policymakers. Achievement level results indicate the degree to which 
student performance meets the standards set for what students should know and be able to 
do at the Basic, Proficient, and Advanced levels. Descriptions of achievement levels 
articulate expectations of performance at each grade level. They are reported as 
percentages of students within each achievement level range, as well as the percentage of 
students at or above the Basic and the percentage at or above Proficient. Results for 
students not reaching the Basic achievement level are reported as below Basic. Results 
are also reported for subgroups of students using demographic data and background 
variables specific to the NAEP Writing Assessment. An individual student’s performance 
cannot be reported based on NAEP results. 

Exhibit 5.1 displays the Governing Board’s generic policy definitions for Basic, 
Proficient, and Advanced achievement that pertain to all NAEP subjects and grades. 
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Exhibit 5.1. Generic Achievement Level Policy Definitions for NAEP 



Achievement 

Level 


Definition 


Advanced 


This level signifies superior performance. 


Proficient 


This level represents solid academic performance for each grade assessed. 
Students reaching this level have demonstrated competency over 
challenging subject matter, including subject-matter knowledge, application 
of such knowledge to real-world situations, and analytical skills appropriate 
to the subject matter. 


Basic 


This level denotes partial mastery of prerequisite knowledge and skills that 
are fundamental for proficient work at each grade. 



There are three components to the NAEP achievement levels: 

• Achievement Level Descriptions 

• Cut scores 

• Examples of students’ responses 
Achievement Level Descriptions 

The Achievement Level Descriptions represent an informed judgment of how good is 
good enough at each grade. Achievement Level Descriptions represent student 
performance on the evaluative criteria specified in the rubrics. 

The full text of the preliminary Achievement Level Descriptions for the NAEP Writing 
Assessment (appendix D) includes descriptions for all three achievement levels at each 
grade. The following text presents an excerpt from the preliminary descriptions for grade 
8 at the Proficient level. 

Grade 8 

Students performing at the proficient level should be able to: 

• Create a mostly competent response to the topic, purpose, and audience 

• Present ideas that support the topic, purpose, and audience and show evidence of 
effective thinking and writing approaches that support development of ideas 

• Use supporting details that are relevant and effective to develop ideas 

• Create a response using logical grouping of ideas and thinking and writing approaches 
that are relevant to the topic, puipose, and audience 

• Use sentences that are usually controlled and that are varied, as appropriate, for the topic, 
puipose, and audience 

• Use word choice that is mostly specific and that is appropriate to communicate 
relationships among ideas 

• Use voice and tone that are usually controlled and appropriate for the topic, purpose, and 
audience 

• Use grammar, usage, and mechanics that are mostly correct but with some distracting 
errors 

The preliminary Achievement Level Descriptions included in the 2011 NAEP Writing 
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Framework will be revised when actual student responses have been collected. These 
preliminary descriptions are included in the framework to guide the development of 

writing tasks of increasing complexity across grade levels and to support the initial stages 
of standard setting. 

After decisions about the implementation of the assessment have been finalized (i.e., 
standard versus non-standard platform, what word processing tools will be enabled, etc.), 
the Governing Board will convene panels of experts to examine the preliminary achieve- 
ment level descriptions and to recommend final Achievement Level Descriptions for each 
grade level. A broadly representative panel of exceptional teachers, educators, and 
professionals will then be convened to engage in a standard setting process to determine 
the cut scores that correspond to these Achievement Level Descriptions. The panelists 
will be trained and will engage in a series of discussions designed to ensure informed 
judgments about mapping cut scores to the assessment. 

Cut Scores 

Cut scores, the second component of reporting on achievement levels, represent the 
minimum score required for performance at each NAEP achievement level. Cut scores 
are reported along with the percentage of students who scored at or above the cut score. 

Examples of Students’ Responses 

The third component of achievement level reporting includes examples of student 
responses on released writing tasks. As in previous NAEP writing assessments, these 
examples provide illustrations of student skills within each level of achievement. In 
addition to examples of responses at each achievement level, the 2011 NAEP Writing 
Assessment will provide examples to demonstrate achievement for each communicative 
purpose at grades 4, 8, and 12 and at each score point. Example responses will be 
annotated to explain the score for the response and salient features of development, 
organization, and language use in relation to the writer’s purpose and audience. 
Significant strengths and weaknesses of the response will be addressed in annotations as 
well. 

Reporting Background Variables 



Background data on students, teachers, and schools are needed to fulfill the statutory 
requirement that NAEP include information, whenever feasible, for groups identified in 
the first section of this chapter (e.g., gender, race/ethnicity). Therefore, students, teachers, 
and school administrators participating in NAEP are asked to respond to questionnaires 
designed to gather demographic information. Information is also gathered from non- 
NAEP sources, such as state, district, or school records. 

In addition to demographic information, background questionnaires include questions 
about variables related to achievement in writing. The variables are selected to be of 
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topical interest, to be timely, and to be directly related to academic achievement and 
current trends and issues in writing. Questions do not solicit information about personal 
topics or information irrelevant to the collection of data on writing achievement. 
Background variables for the 201 1 NAEP Writing Assessment reflect two primary 
interests: computer use (particularly with regard to writing with computers) and the 
number and kinds of opportunities students have to write. NAEP’s ability to show 
similarities and differences among groups in relation to achievement in writing has the 
potential to inform educational policy at the national level as well as in states and school 
districts. For example, data from a question about students’ use of computers might show 
that students who score in the Proficient range compose on the computer more often than 
those who score in the Basic range. 

Recommendations for background variables for the 2011 NAEP Writing Assessment are 
presented in Writing Background Variables Recommendations, but some highlights 
follow. Exhibit 5.2 displays several topics for background variables on technology and 
opportunities to write. 



Exhibit 5.2. Highlights of Background Variables Recommended for the 2011 
NAEP Writing Assessment 



Topics 


Grades 4, 8, and 12 


Students: Computer 
Use 


Frequency and purpose of computer use (e.g., doing homework, 
writing, Internet research, computer games) 


What tools are commonly used for composing and what tools were 
used on the 2011 NAEP Writing Assessment (grades 8 and 12) 


English Language Arts 
(EL A) Teachers: 
Computer Use 


How often and how computers are used in instruction (e.g., to write 
papers, do Internet research, use an online encyclopedia) 


How students are instructed to use computers for writing (e.g., find 
information for writing, generate ideas, compose their first draft) 


Schools: Computer Use 


School expectations or standards for computer proficiency and/or 
technological literacy 


Location and accessibility of computers (e.g., classrooms, labs, 
libraries) 


Students: 

Opportunities to Write 


How often the student is given a specific time period for writing in 
class (not including tests), and how much time is usually allowed 


Kinds of writing students compose on the computer outside of 
school (e.g., e-mail, blogs, instant messaging) 


EL A Teachers: 
Opportunities to Write 


Puiposes for writing taught or assigned 


Participation in professional development related to the teaching of 
writing 


Schools: Opportunities 
to Write 


Existence of and extent to which writing is a school-wide initiative 
(e.g., writing across the curriculum, literacy coaching, etc.) 


Opportunities for professional development in writing 



New Component of NAEP Reporting: Profile of Student Writing 

The 2011 NAEP Writing Framework recommends that a subset of student responses at 
grades 4, 8, and 12 be selected and examined to obtain more in-depth information about 
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what students do in relation to the three domains of writing assessed by NAEP — 
development of ideas, organization of ideas, and language facility and conventions. The 
analyses in the Profile of Student Writing will be reported at the national level only and 
will provide the public, policymakers, and educators with data about important features 
of student writing. The Profile will also include a considerable number of student 
responses that model qualities of the text features and language facility analyzed in 
student responses, along with detailed annotations describing particular features of the 
response. 

While the feasibility of this new component of NAEP reporting is to be determined, the 
Profile should include four components. The first three would include the results of 
studies of a variety of characteristics of student writing. These three analyses are intended 
to provide observations of what students do in their responses, independent of how the 
responses were scored. For example, one analysis might be designed to investigate how 
the communicative purposes for the tasks impact the choices students make in their 
approaches to developing and organizing ideas and in their uses of language. 

The fourth component of the Profile would then link the results of these analyses to 
achievement. In this section of the Profile, data derived from analyses of rhetorical 
features present in responses would be correlated with performance reflected in the 
achievement levels (Basic, Proficient, and Advanced). For example, this component 
might report on achievement levels in relation to students’ selection of form (at grades 8 
and 12) for To Persuade tasks, or on achievement levels in relation to students’ use of 
certain sentence structures. 

Methodology and Reporting of the Profile of Student Writing 



Sampling 

By selecting a nationally representative sample, the results of the analyses can be 
generalized to the whole population. To produce stable estimates of statistical results for 
the Profile and to enable sound conclusions from the data, the national sample would be 
selected according to specific guidelines based on demographic attributes, geographical 
diversity, kinds of schools, scores on responses, and so on. 

Methodology for Rhetorical Analyses 

Analyses of the responses for the first three components of the Profile should be done in 
two stages. In the first stage, an observational study should be conducted to determine 
what specific dimensions of the responses will be studied, and how key questions used to 
guide further investigation will be developed. In the second stage, evidence drawn from 
the responses should be quantified in relation to the key questions. In some cases, 
computer software can be utilized to support analysis of responses at grades 8 and 12. It 
is expected that this approach will be particularly useful in supporting analyses of some 
language features. For example, if types and variety of sentences were analyzed, 
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computer software could be used to efficiently identify and count types of sentences (e.g., 
simple, compound, complex, periodic, etc.) used in responses. Additionally, if the nature 
and extent of errors were analyzed, computer software could be used to identify and 
count most kinds of language errors that may occur in the responses. 

Reporting Results of Rhetorical Analyses 

Because NAEP will have collected information about a variety of demographic factors 
and about students’ writing experiences, including those related to composing on the 
computer, data drawn from analyses of the responses should be aggregated and disaggre- 
gated in many different ways in order to illuminate how students approach the writing 
tasks. Background variables relevant to key questions should also be analyzed to learn 
what information about the students may be related to findings from the quantitative 
analyses of dimensions of writing. For example, background variables might be used to 
better understand how students from different demographic groups or how students who 
report different kinds of writing experiences respond to each communicative purpose 
across tasks at their grade level. 

Reporting Results of Rhetorical Analyses in Relation to Achievement 

The fourth component of the Profile should include information relating the findings 
from the analyses of development and organization of ideas and from the analysis of 
language use to performance based on achievement levels. In this section of the Profile, 
observations about various dimensions of writing analyzed should be correlated with 
performance at the Basic, Proficient, and Advanced levels. Thus, for example, if one 
component of rhetorical analysis were to address what forms students at grades 8 and 12 
choose (for tasks where a choice is given), this section of the Profile could report what 
kinds of forms students at each achievement level use; whether responses at different 
achievement levels tend to display a greater variety of approaches to thinking and writing 
in relation to forms chosen; and whether form impacts text structures differently at each 
achievement level. 

In all components of the Profile, and in particular in the section linking characteristics of 
students’ texts to performance, care will be taken to avoid reporting results that might 
appear to prescribe specific instructional practices. All results reported will be appro- 
priately contextualized in relation to the parameters of the 2011 NAEP Writing Assess- 
ment with regard to time allowed for composing and the nature of the writing task. 

Uses of NAEP Reporting 



The depth and extent of the information available from results of the 2011 NAEP Writing 
Assessment will provide important data during the tenure of the framework. The results 
of the 2011 NAEP Writing Assessment will begin a new trend line: policymakers, 
educators and the public can use data from the assessments as a tool for monitoring 
certain aspects of student achievement in writing over time. NAEP reports will compare 
student performance across states, within the subject area of writing, and among groups 
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of students within the same grade. Long-term achievement trends (e.g., the comparison of 
score performance to previous administrations) can also be reported starting in 2015. 
Because the 2011 NAEP Writing Assessment will measure some writing experiences but 
not all, there will be limitations to the range and scope of information it can produce. 
NAEP publishes data on student performance in relation to various achievement levels 
and demographic subgroups; the information reported does not evaluate results or 
provide conclusive statements about the level of achievement among the nation’s K-12 
students. Furthermore, the 2011 NAEP Writing Assessment is not designed to inform 
instruction — to guide how writing is taught — only to measure a representative sample of 
the student population at grades 4, 8, and 12 and students’ performance within the 
assessment context outlined in this framework. 

The important components of NAEP reporting are summarized in Exhibit 5.3. 



Exhibit 5.3. Summary of NAEP Reporting Components 







Key Characteristics 






Elements released to the public include: 




How 


• Printed summary report known as The Nation’s 




Information is 


Report Card 




Reported 


• Dedicated website: http://nationsreportcard.gov 
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• Performance of various subgroups at the 
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Level Descriptions 
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What 


Types of background variables distributed to students, teachers, 
and schools: 




Information is 
Gathered 


• Demographics 

• Computer use (especially in writing) 

• The number and kinds of opportunities to write 
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Components of the Profile include: 
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• Analysis of students’ development of ideas 
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Profile of 
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• Analysis of students’ language facility and 
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• Exploratory Analysis: relationship between 
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performance and attributes of writing 
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Conclusion 



The goal of the 2011 NAEP Writing is to meet the demands of assessing young 
people’s ability to write to communicate in the 21 st century. 

The 2011 NAEP Writing Assessment represents a way forward for the large-scale 
assessment of writing. By focusing on the kinds of purpose-centered writing most 
common to K-12 and postsecondary settings, and by emphasizing the engagement of 
clearly defined audiences, the assessment better emulates many real-world writing 
situations. The 2011 assessment’s emphasis on students’ rhetorical flexibility will 
encourage a rich variety of response approaches — students will have the freedom to and 
will be expected to draw upon many critical thinking and writing approaches to develop, 
organize, and articulate ideas. 

The 2011 NAEP Writing Assessment reflects emerging definitions of what “effective 
communication” means. Because students will compose with word processing software 
at grades 8 and 12, and because common word processing tools that support writing will 
be enabled, the assessment offers a writing environment reflective of the way students 
most commonly compose in the 21 st century. The on-demand writing scenarios that will 
be presented in the 201 1 NAEP Writing Assessment are similar to many writing 
situations in academic and professional settings and provide a relevant survey of how 
students perform under time constraints. 

The 2011 NAEP Writing Assessment seeks new ways of understanding student writing 
achievement. By using holistic rubrics designed specifically to assess performance in 
relation to particular communicative purposes for writing, and by training scorers to 
recognize the effectiveness of a variety of kinds of responses, the assessment design 
encourages writing that moves beyond prescriptive or formulaic qualities. In addition, 
new information on the kinds of writing students and teachers are composing in and out 
of school, as well as the role computers play in that writing (and writing instruction) will 
be gathered. Further, analysis of a nationally representative subset of responses will 
provide a deeper understanding of important features of students’ writing and of 
connections between student performance and specific components of the assessment. 
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Appendix A 

Glossary of Terms 

This glossary provides brief definitions of terms used throughout the 2011 NAEP Writing 
Framework. The terms are defined according to their use in the framework. The list 
includes terms that relate to types of reading materials, text structures and features, 
techniques of author’s craft, and other key terms. 

Accessibility: Content and design considerations that ensure all student populations can 
demonstrate what they know and can do on the 201 1 NAEP Writing Assessment. 

Achievement Level Descriptions: Statements established by the National Assessment 
Governing Board that define what students know and can do at each achievement level of 
writing (Basic, Proficient, and Advanced). 

Accommodations: Changes to test materials or procedures (e.g., large print booklets, 
extended time) that allow students with disabilities and English language learners an 
equal opportunity to demonstrate what they know and can do. 

Analyzing: An approach to thinking and writing that breaks down a large topic into 
logical parts, which can then be extensively examined individually or in relation to a 
broad subject as a whole. 

Approaches to thinking and writing: Methods used on a writing task (e.g., analyzing, 
describing, narrating, synthesizing) to support the development and organization of ideas 
in relation to the purpose and audience specified. 

Arguing: An approach to thinking and writing that attempts to prove or disprove by 
using a range of strategies, such as giving reasons, supporting and organizing evidence 
(e.g., analogies, illustrations), and/or by considering pros and cons on a subject. 

Assessment: The evaluation of students’ academic skills to determine what they know 
and can do. 

Framework: For each NAEP subject, a framework is developed to describe the design of 
the assessment, the content to be assessed by NAEP, how that content will be measured, 
and how results will be reported. 

Assessment Specifications: The document that specifies how framework guidelines are 
to be implemented in relation to the content, design, evaluation, and reporting of a NAEP 
subject assessment. 

Audience: The specified or clearly implied person(s) on a 201 1 NAEP Writing 
Assessment task to whom writers address their responses. 
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Background variables: Student, teacher, and school background questionnaires used to 
collect data that may support analysis of student performance on an assessment. 

Coherence: The consistent relationship of parts and ideas in a piece of writing, helping 
the reader understand the writer’s purpose and his or her argument. 

Communicative purposes: The aim or goal of a writing task. In the context of NAEP, 
three communicative purposes are assessed: to persuade; to explain; and to convey 
experience, real or imagined. 

Complexity: Refers to a) the level with which writing addresses the implications, 
complications, and multiple dimensions of a topic or issue, and b) the increase in 
difficulty of tasks and expectations for writing across grades. 

Computer-based testing: The administration of any assessment on computers. 

Construct (noun): The articulation of all components of the assessment as they relate to 
what the assessment is designed to measure. 

Contextual information: Information provided in the task that supports understanding of 
the situation or topic to be addressed in a response. Contextual information on some 
NAEP Writing Assessment tasks will also include visual stimuli and reading passages. 

Constructed response: A type of assessment task that requires students to produce their 
own answer rather than selecting from a given list (e.g., multiple choice). On the 2011 
NAEP Writing Framework, the method of constructed response is the completion of 
written composition. 

Conventions: Commonly accepted rules and guidelines for formal written language 
concerning grammar and usage, as well as mechanics (e.g., punctuation, capitalization, 
and spelling). 

Convey experience, real or imagined: A communicative purpose in which students 
engage the reader in an event or perspective, real or imagined. 

Cut score: The minimum score required to attain a NAEP achievement level (e.g., Basic, 
Proficient, or Advanced). 

Describing: An approach to thinking and writing that depicts a person, object, or idea in 
ways that appeal to the senses of the writer’s audience. 

Development of ideas: The use of evidence, support, and approaches in expressing a 
purpose so that an audience can comprehend the writer’s understanding of a topic or 
issue. 
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Direct writing assessment: The measurement of students’ writing abilities by their 
performance on a writing task. 



Distracting errors: Mistakes in language conventions and/or usage that prevent a reader 
from fully understanding the writer’s ideas or message. 

Domain: A broad feature of all writing (e.g., language facility), comprised of specific 
and important components that can be used to assess student performance within this 
feature. 

English language learner: A student who is not a native English speaker and is still in 
the process of English language acquisition. 

Evaluating: An approach to thinking and writing that defines and justifies the 
significance, value, or quality of an object or idea. 

Explain: A communicative purpose in which a writer’s aim is to make a subject, issue, or 
concept understandable to the reader. 

Field testing: The process by which tasks under consideration for an assessment are 
determined to be accessible, comparable, and effective in discriminating among various 
levels of achievement or ability. Field tests are designed to simulate an actual assessment. 

Focus: The emphasis on and/or consistent development of a main point, controlling idea, 
or theme. 

Form: Types of text readily identifiable because of common organizational patterns or 
language features. 

Grammar: A set of rules that define how a language is structured and communicated, 
thereby creating a standard language that writers commonly follow. 

Holistic scoring rubric: A guide for scorers to evaluate writing performance, focusing 
on assessing performance across multiple domains — e.g., development, organization, 
language facility and conventions — to evaluate overall performance. Scorers evaluating 
responses on the 201 1 NAEP Writing Assessment will use a holistic rubric with a scale 
of 1-6. 



Inclusion criteria: The parameters used to determine whether a student with special 
needs or an English Language Learner is eligible for participation on a NAEP 
assessment. 

Individual Education Program (IEP): The document that outlines an appropriate 
instructional program and relevant accommodations for a student with special needs. 
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Interpreting: An approach to thinking and writing that explains the meaning or 
significance of an idea not readily apparent from initial reading, discussion, or common 
understanding. 

Large-scale assessment: Standardized assessment program designed to evaluate the 
achievement of large groups of students. 

Mechanics: Conventions of spelling, capitalization, and punctuation. 

NAEP special study: Research conducted by the National Assessment of Educational 
Progress that provides new information on NAEP assessments or a subject matter NAEP 
assesses. 

Narrating: An approach to thinking and writing that presents events in a meaningful 
order — often chronological — that imparts experience, knowledge, or description. 

On-demand writing: The assessment of students’ writing abilities in a limited time 
frame. 

Persuade: A communicative purpose in which a writer attempts to convince an audience 
of one’s point of view or to move them to action. 

Reading passages: Text (e.g., such as a quotation or short excerpt from a book or article) 
included in a writing task to stimulate and support the writer’s response. 

Reflecting/Questioning: An approach to thinking and writing in which a writer 
demonstrates self-examination or contemplation in his or her response. 

Rhetorical flexibility: The writer’s ability to adapt his/her ideas, organization, syntax, 
word choice, and other conventions of writing depending upon the purpose and audience 
of the writing task. 

Rubric: A scoring guide that notes what specific features of student learning will be 
assessed and the expected level of performance at each scorepoint. A rubric is used to 
guide evaluators’ assignment of numerical scores to different levels of performance. 

Scaffolding: In a writing task, the clarity and quality of contextual information provided 
to support and guide the writer’s response. 

Scale score: A score derived from student responses to NAEP assessment items that 
summarizes the overall level of performance attained by a group of students. This 
score — presented as a number on a set scale — provides information about what a 
particular aggregate of students (e.g., grade 4 students) know and can do in writing. 

Scorers: The evaluator or rater of responses to NAEP writing tasks. 
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Section 504 plan: The document that outlines a program of instructional services to 
assist students with special needs who are in mainstream educational settings. 

Sentence structure: The ways in which sentences are organized and composed. 

Sentence variety: The use of different lengths and kinds of sentence structures to engage 
an audience and support the clear expression of ideas. 

Students with disabilities: Those students identified as having a disability, specified in 
an Individualized Education Program or a Section 504 Plan, and potentially in need of 
accommodations to complete the 2011 NAEP Writing Assessment. 

Style: Elements of language (e.g., word choice, sentence variety) that the writer utili z es 
to produce the most effective presentation of ideas in relation to his or her purpose and 
audience. 

Summarizing: An approach to thinking and writing that expresses the main points of one 
or several resources, including readings, research findings, events, the writer’s own ideas, 
etc. 

Syntax: The order of words, phrases, and clauses within a sentence. 

Synthesizing: An approach to thinking and writing that combines different ideas or 
information into a coherent whole so a new understanding of a subject or issue is 
conveyed. 

Text: A piece of writing. 

Text structure: The organizational arrangement of written information to convey the 
purpose of a piece of writing to an audience (e.g., directions, problem-solution). 

Tone: The writer’s attitude toward the subject matter of the writing and/or the audience 
reading his or her work. 

Trend line: For the purposes of NAEP, a long-term measure of student achievement as it 
corresponds to other factors (e.g., time, demographics, etc.). 

Usage: Established conventions of word choice and phrasing. 

Visual stimuli (in a writing task): Pictures, drawings, charts, graphs, or other images 
used in a writing task to stimulate and support the writer’s response. 

Voice: A writer’s ability to convey attitude, personality, and/or character appropriate to 
the writing situation. 
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Word choice (diction): The use of vocabulary appropriate for the purpose and audience 
of a writing task. 

Word processing software with commonly available tools: Computer applications that 
allow users to write, edit, and produce texts. “Commonly available tools” refers to the 
extensions built into this software that help writers modify or revise their text documents. 

Writing process: A series of overlapping and recursive processes (e.g., prewriting, 
drafting, revising, and editing) a writer often moves among when developing a piece of 
writing. 

Writing task: Assessment item designed to elicit extended written response that can be 
evaluated using specified criteria. 
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Appendix B 



Example Tasks 1 



Grade 4 Examples 



To Persuade 




Imagine that students at your school are going to select a new school mascot. A mascot is 
an animal or object used to represent a group. For example, many sports teams have 
mascots. 

Four choices are being considered as your school’s mascot: Tigers, Rising Stars, 
Dolphins, and Rockets. You have been asked to choose one of the four mascots and to 
support your choice in a letter to the school principal. 

Write a letter to your principal convincing him or her that your choice should be the 
school mascot. Be sure to include reasons and examples in your letter. 



1 In order to ensure the accessibility of all tasks to all students, accommodations — such as large print, 
Braille, and audio — may be provided to students who qualify. Information on accommodations for the 2011 
NAEP Writing Assessment is provided in the specifications document. 
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To Explain 



Imagine your teacher has chosen you to help a new student who will soon be attending 
your school. To prepare the new student, your teacher has asked you to write a letter to 
the new student explaining what your school is like so that he will know what to expect 
on his first day. 

Write a letter to your future classmate explaining what your school is like. Be sure to 
include details and examples in your letter that will help him learn about your school. 

To Convey Experience, Real or Imagined 

Your school would like to help students think about how a person’s actions can make a 
difference to others. The school newspaper is planning to publish stories about times 
when students helped someone or when someone else helped them. 

Write a story for the school newspaper about a time when you helped someone or a time 
when someone helped you. Be sure to include details in your story that convey your 
experience to your readers. 

Grade 8 Examples 



To Persuade 

Your school wants to persuade new eighth grade students to participate in school or 
community activities, sports, or clubs by publishing a brochure about the school’s 
extracurricular offerings. Current students have been asked to write about a particular 
sport, club, or activity they participate in or about some other activity they think new 
eighth students should get involved in. 

Compose a piece of writing to persuade new eighth grade students to participate in the 
sport, club, or activity you have chosen. Be sure to include reasons and examples that will 
persuade new eighth grade students to participate in the activity you have chosen to write 
about. 



NAEP 201 1 Writing Framework 
60 




To Explain 



Your school is creating a “time capsule,” a box containing objects and writings from the 
current year that will be opened by eighth graders in the year 2050. All current eighth 
graders have been asked to add to the time capsule by writing about what they think life 
in the United States might be like in 2050. 

Compose a piece of writing for the time capsule explaining to future students what you 
think life in the United States will be like in 2050. Be sure to explain your ideas by using 
details and examples. The information below shows predictions about the future that you 
may consider using in your writing. 



A magazine for young adults published the following survey results: 



Survey of 8 ,r ' Grade Students' Predictions for 2050 




Predicted Events 



The following chart appeared in the Careers section of your load newspaper: 



Predicted Job Trends 


Increase 


Decrease 


• Teacher 


• Farmer 


• Computer Technician 


• Salesperson 


• Scientist 


• Lawyer 
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To Convey Experience, Real or Imagined 



Teen Life , a magazine for young adults, has announced a writing contest for middle 
school students. The theme of the contest is “Achieving Goals.” The magazine has 
published the following contest instructions: 

Teen Life wants to hear about the experiences of young people who have achieved goals. To 
enter, write about a memorable moment in your life when you achieved a goal you set for 
yourself. All successful responses will need to clearly convey the experience of achieving a goal 
so that the reader can fully understand the experience and its importance. The staff of Teen Life 
will select a winner, which will be published in next month ’s issue. 

Write a response for the contest, describing an experience of achieving a goal and the 
importance of that experience to your life. Be sure to include details in your response that 
help readers understand your experience and its importance. 



Grade 12 Examples 



To Persuade 

Scientists recently announced that a state park in your state contains large amounts of 
oil — a substance that can be converted into gasoline for cars or used to heat homes. Your 
state legislature is now debating whether to allow energy companies access to the 
protected land in this park in order to drill for the oil. In a speech to residents of the state, 
one of the state legislators identifies two sides of the debate: 

“The debate over whether the state should allow energy companies to drill for petroleum 
on protected land is important to this state’ s future. Some in the state legislature believe 
that allowing access to lands currently protected would benefit our economy, creating 
thousands of jobs and lowering the costs of goods and services. Others, however, believe 
it is more important to protect our natural environment and support a high quality of life 
for citizens of this state. ” 

The state legislature has called on citizens of the state to express their opinions about this 
issue. Take a position and write a response persuading members of your state legislature 
to support your position on whether or not protected land in your state should be opened 
to energy companies for drilling. 

To Explain 

Members of your community, including local leaders and the mayor, are concerned about 
civic awareness and town pride. In order to open a dialogue with area residents, your 
local newspaper is inviting residents to respond to a question civic leaders have debated: 
“What makes a good community?” The newspaper wants those who respond to define a 
good community and to explain what elements are needed to create a good community. 



NAEP 201 1 Writing Framework 
62 




Responses to this question will be read by members of the city council, including the 
mayor, and used to support their efforts to improve civic awareness and town pride. 

Write a response for the newspaper in which you define a good community and explain 
what elements make a good community. Be sure to use specific examples and details to 
explain your ideas. 

To Convey Experience, Real or Imagined 

As part of an application for a job or college, you have been asked to consider how the 
following quotation is related to your high school experience. 

“Experience is not what happens to you; it is what you do with what happens to you. ” 

— Aldous Huxley 

Write a response to this quotation in which you relate a high school experience that 
shows how you have grown as a student since starting high school. Be sure to use details 
that convey the experience to readers of your application. 
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Appendix Cl 



Preliminary Holistic Scoring Guide for 

To Persuade 

Score = 6 Responses in this range demonstrate effective skill in responding to the writing 
task. All elements of the response are well controlled and effectively support the writer’s 
purpose, audience, and form. 

• The response formulates a clear position that recognizes and acknowledges multiple 
significant aspects of the issue and insightfully addresses the complexities of the issue. 
The response demonstrates insight by fully addressing other perspectives, by fully 
evaluating implications of the writer’s position, and/or by using affective arguments that 
are consistently persuasive. 

• The response provides strong persuasive reasons and evidence to support the writer’ s 
position. Approaches to the development of ideas (e.g., summarizing, narrating, etc.) are 
used skillfully to support the persuasive purpose. 

• Ideas are clearly focused on the topic throughout the response. Organization demonstrates 
a logical, well-executed progression of ideas that effectively supports the persuasive 
puipose and is relevant to the writer’s approaches to organization (e.g., analyzing, 
evaluating, narrating, etc.). Transitions effectively convey relationships among ideas. 

• Sentence structure is well controlled and varied to communicate relationships among 
ideas. Word choice is precise and evaluative and supports the persuasive purpose. Voice 
and tone are well controlled and effective for the writer’s purpose and audience. 

• Though there may be a few minor errors in grammar, usage, and mechanics, meaning is 
clear throughout the response. 

Score = 5 Responses in this range demonstrate competent skill in responding to the writing 
task. Elements are usually well controlled and clearly support the writer’s purpose, 
audience, and form. 

• The response formulates a position that usually recognizes and acknowledges multiple 
significant aspects of the issue but may not fully address some of the complexities of the 
issue. The response demonstrates some insight by acknowledging and partially 
addressing other perspectives, by evaluating some implications of the writer’s position, 
and/or by using affective arguments that are usually persuasive. 

• The response usually provides persuasive reasons and evidence to support the writer’s 
position. Approaches to the development of ideas are usually used skillfully to support 
the persuasive purpose. 

• Ideas are usually focused on the topic. Organization is clear and may demonstrate a 
logical progression of ideas that supports the persuasive purpose and is relevant to the 
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writer’s approaches to organization. Transitions clearly convey relationships among 
ideas. 

• Sentence structure is well controlled to communicate relationships among ideas and 
varied as appropriate for the writer’s purpose. Word choice is usually precise and 
evaluative and usually supports the persuasive purpose. Voice and tone are usually 
controlled and effective for the writer’ s puipose and audience. 

• Grammar, usage, and mechanics are usually correct with a few distracting errors, but 
meaning is clear. 

Score = 4 Responses in this range demonstrate adequate skill in responding to the writing 
task. Most elements are controlled and support the writer’s purpose, audience, and form. 

• The response takes a position and may acknowledge significant aspects of the issue. The 
response demonstrates some understanding of other perspectives and may evaluate some 
implications of the writer’s position. If affective arguments or examples are used they are 
persuasive. 

• While details and examples provide adequate evidence to support the writer’s position, 
their development may be somewhat uneven. Approaches to the development of ideas are 
adequate, but their relevance to the persuasive purpose may not always be clear. 

• Ideas are usually focused on the topic, and an organizational structure is evident. Ideas 
are logically grouped and adequately reflect the writer’s use of relevant approaches to 
organization. Relationships among ideas are mostly clear. 

• Sentence structure is adequately controlled and somewhat varied to communicate 
relationships among ideas. Word choice is clear, often evaluative, and adequately 
supports the persuasive purpose. Voice and tone are mostly controlled and usually 
effective for the writer’s puipose and audience. 

• Grammar, usage, and mechanics are mostly correct with some distracting errors, but 
meaning is clear. 

Score = 3 Responses in this range demonstrate developing skill in responding to the 
writing task. Some elements are controlled and provide some support for the writer’s 
purpose, audience, and form. 

• The response states a position but addresses only some of the aspects of the issue. The 
response shows little understanding of other perspectives, although most ideas are 
relevant to the persuasive purpose. 

• Some relevant reasons and evidence for the writer’s position are used, but they are not 
developed enough to be convincing, or they may be unevenly developed. Approaches to 
development of ideas are evident, but they may not be clearly relevant to the persuasive 
puipose. 

• Most ideas are focused on the topic. The response uses a simple organizational structure, 
and, for the most part, ideas are logically grouped. There is some evidence of the writer’s 
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use of approaches to organization, but they may not be clearly relevant, or they may be 
confusing. Relationships among ideas are sometimes unclear. 

• Sentence structure is usually correct and there may be a little sentence variety to 
communicate relationships among ideas. Word choice is usually clear and sometimes 
evaluative but at times may not be appropriate for the writer’s purpose. Voice and tone 
show some understanding of what is appropriate for the writer’s purpose and audience. 

• Grammar, usage, and mechanics are mostly correct but with some distracting errors that 
may occasionally impede understanding. 

Score = 2 Responses in this range demonstrate marginal skill in responding to the writing 
task. Elements are sometimes controlled but provide weak support for the writer’s purpose, 
audience, and form. 

• The response states a position and provides a few reasons to support the writer’s position. 

• A few reasons may be given, but they are not developed enough to be convincing. If 
details and examples are present, they are brief, general, or inadequately developed, and 
they may not be clearly relevant to the persuasive purpose. There may be minimal 
evidence of relevant approaches to the development of ideas. 

• Some ideas may not be clearly focused on the topic. The response shows an attempt to 
organize thoughts by grouping ideas, and there may be minimal evidence of relevant 
approaches to organization. However, relationships among ideas are often illogical or 
unclear. 

• Sentence structure is sometimes correct, but there is little, if any, sentence variety. Word 
choice is rarely specific and does little to support the persuasive purpose. Voice and tone 
show little understanding of what is appropriate for the writer’ s purpose and audience. 

• Grammar, usage, and mechanics are usually correct but with many distracting errors that 
impede understanding. 

Score = 1 Responses in this range demonstrate little or no skill in responding to the writing 
task. Elements are seldom controlled and provide almost no support for the writer’s 
purpose, audience, and form. 

• The response may state a position and may give a few simplistic reasons to support the 
writer’ s position. 

• If any details or examples are present, they are brief, general, undeveloped, or not 
relevant to the persuasive purpose. 

• The response shows an attempt to organize thoughts by grouping ideas, but groupings are 
illogical and there is little or no evidence of relevant approaches to organization. 
Relationships among ideas are mostly unclear. 

• Sentence structure is often incorrect; word choice is often unclear and inappropriate; and 
there is little or no control of appropriate voice and tone. 
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Score 



Grammar, usage, and mechanics are sometimes correct but with frequent distracting 
errors that often impede understanding. 

The response may be too brief to support a sound judgment about the development of 
ideas, organization, or language facility and conventions. 

0 Unscorable: Response is too brief to score; not written in English; off topic; or 
illegible. 
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Appendix C2 



Preliminary Holistic Scoring Guide for 

To Explain 

Score = 6 Responses in this range demonstrate effective skill in responding to the writing 
task. All elements of the response are well controlled and effectively support the writer’s 
purpose, audience, and form. 

• The response provides a thoughtful and insightful explanation of the subject by fully 
examining the topic as a whole, by identifying and fully discussing significant parts of 
the subject, and/or by evaluating and fully discussing the importance of the parts. 

• The explanation maintains an effective balance between broad assertions and well chosen 
general and specific details and examples to fully support understanding. Approaches to 
the development of ideas (e.g., analyzing, evaluating, narrating, etc.) are used skillfully to 
support the clarity of the explanation. 

• Ideas are clearly focused on the topic throughout the response. Organization demonstrates 
a logical, well-executed progression of ideas that supports the clarity of the explanation 
and is relevant to the writer’s approaches to organization (e.g., summarizing, narrating, 
etc.). Transitions effectively convey relationships among ideas. 

• Sentence structure is well controlled and varied to communicate relationships among 
ideas. Word choice is specific, precise, and evaluative and supports the clarity of the 
explanation. Voice and tone are well controlled and effective for the writer’s purpose and 
audience. 

• Though there may be a few minor errors in grammar, usage, and mechanics, meaning is 
clear throughout the response. 

Score = 5 Responses in this range demonstrate competent skill in responding to the writing 
task. Elements are usually well controlled and clearly support the writer’s purpose, 
audience, and form. 

• The response provides a clear explanation of the subject by examining the topic as a 
whole, identifying and discussing various parts of the subject, and/or by evaluating and 
discussing the importance of those parts. 

• The explanation usually maintains an effective balance between broad assertions and 
general and specific details and examples to support the clarity of the explanation. 
Approaches to the development of ideas are usually used skillfully to support the clarity 
of the explanation. 

• Ideas are usually focused on the topic. Organization is clear and may demonstrate a 
logical progression of ideas that supports the clarity of the explanation and is relevant to 
the writer’s approaches to organization. Transitions clearly convey relationships among 
ideas. 
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• Sentence structure is well controlled to communicate relationships among ideas and 
varied as appropriate for the writer’s purpose. Word choice is usually specific and 
precise, and it usually supports the clarity of the explanation. Voice and tone are usually 
controlled and effective for the writer’ s purpose and audience. 

• Grammar, usage, and mechanics are usually correct with a few distracting errors, but 
meaning is clear. 

Score = 4 Responses in this range demonstrate adequate skill in responding to the writing 
task. Most elements are controlled and support the writer’s purpose, audience, and form. 

• The response provides an explanation by addressing most parts of the subject and may 
include an evaluation of the importance of some of these parts. 

• The explanation maintains an adequate balance between broad assertions and specific 
examples and details. While details and examples adequately support the clarity of the 
explanation, their development may be somewhat uneven. Approaches to the develop- 
ment of ideas are adequate, but their relevance to the explanation may not always be 
clear. 

• Ideas are usually focused on the topic and an organizational structure is evident. Ideas are 
logically grouped and adequately reflect the writer’s use of relevant approaches to 
organization. Relationships among ideas are mostly clear. 

• Sentence structure is adequately controlled and somewhat varied to communicate 
relationships among ideas. Word choice is sometimes specific and adequately supports 
the clarity of the explanation. Voice and tone are mostly controlled and usually effective 
for the writer’s purpose and audience. 

• Grammar, usage, and mechanics are mostly correct with some distracting errors, but 
meaning is clear. 

Score = 3 Responses in this range demonstrate developing skill in responding to the 
writing task. Some elements are controlled and provide some support for the writer’s 
purpose, audience, and form. 

• The response provides some explanation of the subject. 

• The explanation may provide both broad assertions and general and specific examples, 
but the balance may be uneven and only somewhat controlled. Some relevant details and 
examples are used, but they are not developed enough to support the explanation, or they 
may be unevenly developed. Approaches to the development of ideas are evident, but 
they may not be clearly relevant to the explanation. 

• Most ideas are focused on the topic. The response uses a simple organizational structure, 
and for the most part, ideas are logically grouped. There may be some evidence of 
approaches to organization, but they may not be clearly relevant, or they may be 
confusing. Relationships among ideas are sometimes unclear. 
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• Sentence structure is usually correct and there may be a little sentence variety to 
communicate relationships among ideas. Word choice is usually clear and sometimes 
specific, but at times it may not be appropriate for the writer’s purpose. Voice and tone 
show some understanding of what is appropriate for the writer’s purpose and audience. 

• Grammar, usage, and mechanics are mostly correct but with some distracting errors that 
may occasionally impede understanding. 

Score = 2 Responses in this range demonstrate marginal skill in responding to the writing 
task. Elements are sometimes controlled but provide weak support for the writer’s purpose, 
audience, and form. 

• The response provides a little explanation of the subject. 

• A balance between broad assertions and general and specific examples may not be 
evident. If details and examples are present, they are brief, general, or inadequately 
developed, and they may not be clearly relevant to the explanation. There may be 
minimal evidence of relevant approaches to the development of ideas. 

• Some ideas may not be clearly focused on the topic. The response shows an attempt to 
organize thoughts by grouping ideas, and there may be minimal evidence of approaches 
to organization. However, relationships among ideas are often illogical or unclear. 

• Sentence structure is sometimes correct, but there is little, if any, sentence variety. Word 
choice is rarely specific and does little to support the clarity of the explanation. Voice and 
tone show little understanding of what is appropriate for the writer’s puipose and 
audience. 

• Grammar, usage, and mechanics are usually correct but with many distracting errors that 
impede understanding. 

Score = 1 Responses in this range demonstrate little or no skill in responding to the writing 
task. Elements are seldom controlled and provide almost no support for the writer’s 
purpose, audience, and form. 

• The response may provide little or no explanation of the subject. 

• If any details or examples are present, they are brief, general, undeveloped, or not 
relevant to the explanation. 

• The response shows an attempt to organize thoughts by grouping ideas, but groupings are 
illogical and there is little or no evidence of relevant approaches to organization. 
Relationships among ideas are mostly unclear. 

• Sentence structure is often incorrect; word choice is often unclear and inappropriate; and 
there is little or no control of appropriate voice and tone. 

• Grammar, usage, and mechanics are sometimes correct but with frequent distracting 
errors that often impede understanding. 
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Score 



The response may be too brief to support a sound judgment about the development of 
ideas, organization, or language facility and conventions. 



0 Unscorable: Response is too brief to score; not written in English; off topic; or 
illegible. 
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Appendix C3 



Preliminary Holistic Scoring Guide for 
To Convey Experience, Real or Imagined 

Score = 6 Responses in this range demonstrate effective skill in responding to the writing 
task. All elements of the response are well controlled and effectively support the writer’s 
purpose, audience, and form. 

• The response effectively conveys the significance of the experience, either explicitly or 
implicitly, and conveys the complexities of the experience, whether real or imagined. 

• Well chosen examples and sensory details, if appropriate, are effectively used to illustrate 
and recreate the experience for the audience. Approaches to the development of ideas 
(e.g., narrating, describing, analyzing, etc.) are used skillfully to convey the experience. 

• Ideas are clearly focused on the topic throughout the response. Organization demonstrates 
a logical, well-executed progression of ideas that effectively conveys the experience and 
is relevant to the writer’s approaches to organization (e.g., summarizing, narrating, etc.). 
Transitions effectively convey relationships among ideas. 

• Sentence structure is well controlled and varied to communicate relationships among 
ideas. Word choice is connotative, specific, and precise and effectively conveys the 
experience. Voice and tone are well controlled and effective for the writer’s purpose and 
audience. 

• Though there may be a few minor errors in grammar, usage, and mechanics, meaning is 
clear throughout the response. 

Score = 5 Responses in this range demonstrate competent skill in responding to the writing 
task. Elements are usually well controlled and clearly support the purpose, audience, and 
form. 

• The response clearly conveys the significance of the experience, either explicitly or 
implicitly, and conveys some complexities of the experience, whether real or imagined. 

• Effective examples and sensory details, if appropriate, usually illustrate and recreate the 
experience for the audience. Approaches to the development of ideas are usually used 
skillfully to convey the experience. 

• Ideas are usually focused on the topic. Organization is clear and may demonstrate a 
logical progression of ideas that supports the writer’s purpose and is relevant to the 
writer’s approaches to organization. Transitions clearly convey relationships among 
elements of the experience. 

• Sentence structure is well controlled to communicate relationships among ideas and 
varied as appropriate for the writer’s purpose. Word choice is usually connotative, 
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specific, and precise, and it usually supports the writer’s purpose. Voice and tone are 
usually controlled and effective for the writer’s purpose and audience. 

• Grammar, usage, and mechanics are usually correct with a few distracting errors, but 
meaning is clear. 

Score = 4 Responses in this range demonstrate adequate skill in responding to the writing 
task. Most elements are controlled and support the intended purpose, audience, and form. 

• The response adequately conveys the significance of the experience, either explicitly or 
implicitly, and may convey some complexities of the experience, whether real or 
imagined. 

• Some examples and sensory details, if appropriate, are used to illustrate experience, but 
they may need to be more developed or more may be needed to support the writer’s 
purpose. Approaches to the development of ideas are adequate, but their relevance to the 
writer’ s puipose may not always be clear. 

• Ideas are usually focused on the topic and an organizational structure is evident. Elements 
are logically grouped and adequately reflect the writer’s use of relevant approaches to 
organization. Relationships among elements of the experience are mostly clear. 

• Sentence structure is adequately controlled and somewhat varied to communicate 
relationships among ideas. Word choice is often connotative and specific, and it 
adequately supports the experience being conveyed. Voice and tone are mostly controlled 
and usually effective for the writer’ s puipose. 

• Grammar, usage, and mechanics are mostly correct with some distracting errors, but 
meaning is clear. 

Score = 3 Responses in this range demonstrate developing skill in responding to the 
writing task. Some elements are controlled and provide some support for the writer’s 
purpose, audience, and form. 

• The response conveys some elements of the significance of the experience and may 
demonstrate a little awareness of the complexities of the experience. 

• Some examples and sensory details, if appropriate, are used, but they are not developed 
enough to support the writer’ s puipose, or they may be unevenly developed. Approaches 
to the development of ideas are evident, but they may not be clearly relevant to the 
writer’ s purpose. 

• Most ideas are focused on the topic. The response uses a simple organizational structure 
and for the most part elements of the experience are logically grouped. There may be 
some evidence of approaches to organization, but they may not be clearly relevant, or 
they may be confusing. Relationships among elements of the experience are sometimes 
unclear. 
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• Sentence structure is usually correct and there may be a little sentence variety to 
communicate relationships among ideas. Word choice is usually clear and sometimes 
connotative and specific, but at times it may not be appropriate for the writer’s puipose. 
Voice and tone show some understanding of what is appropriate for the writer’s purpose 
and audience. 

• Grammar, usage, and mechanics are mostly correct but with some distracting errors that 
may occasionally impede understanding. 

Score = 2 Responses in this range demonstrate marginal skill in responding to the writing 
task. Elements are sometimes controlled but provide weak support for the writer’s purpose, 
audience, and form. 

• The response conveys a few elements of the experience. 

• If details and examples are present, they are brief, general, or inadequately developed, 
and they may not be clearly relevant to the writer’s puipose. There may be minimal 
evidence of the use of relevant approaches to the development of ideas. 

• Some ideas may not be clearly focused on the topic. The response shows an attempt to 
organize the elements of the experience, and there may be minimal evidence of relevant 
approaches to organization. However, relationships among ideas are often illogical or 
unclear. 

• Sentence structure is sometimes correct, but there is little, if any, sentence variety. Word 
choice is rarely specific and does little to convey the experience. Voice and tone show 
little understanding of what is appropriate for the writer’s purpose and audience. 

• Grammar, usage, and mechanics are usually correct but with many distracting errors that 
impede understanding. 

Score = 1 Responses in this range demonstrate little or no skill in responding to the writing 
task. Elements are seldom controlled and provide almost no support for the purpose, 
audience, and form. 

• The response conveys few elements of the experience. 

• If any details or examples are present, they are brief, general, undeveloped, or not 
relevant to the writer’s purpose. 

• The response shows an attempt to organize thoughts by grouping ideas, but groupings are 
illogical and there is little or no evidence of relevant approaches to organization. 
Relationships among elements are mostly unclear. 

• Sentence structure is often incorrect; word choice is often unclear and inappropriate; and 
there is little or no control of appropriate voice and tone. 

• Grammar, usage, and mechanics are sometimes correct but with frequent distracting 
errors that often impede understanding. 
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• The response may be too brief to support a sound judgment about the development of 
ideas, organization, or language facility and conventions. 

Score = 0 Unscorable: Response is too brief to score; not written in English; off topic; or 
illegible. 
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Appendix D 



2011 NAEP Writing Preliminary Achievement Level 

Descriptions 

NAEP Achievement Level Descriptors define what students should know and be able to 
do at three levels: Basic, Proficient, and Advanced. The matrices in Exhibits 6.1 to 6.3 
below correlate these three achievement levels to expectations of performance on 
important components of writing. Basic, Proficient, and Advanced levels apply to all 
students assessed at grades 4, 8, and 12; however, writing achievement at each of these 
levels will differ at each grade and cannot be compared across grades. To support 
understanding of the Achievement Level Descriptions, student responses should be used 
to define the expectations expressed at each grade level (e.g., “adequate” or “well- 
developed”), and should also function as a demonstration of increasing levels of 
achievement across the grades. 

*Note: The term “thinking and writing approaches” in the descriptions below refers to evidence of relevant 
approaches for development and organization (e.g., analyzing, evaluating, narrating, describing, etc.). 



Exhibit 6.1. Preliminary Basic Achievement Level Descriptions 



Grade 4 


Grade 8 


Grade 12 


Students performing at the Basic level should be able to: 


Create a somewhat appropriate 
response to the topic, purpose, 
audience, and specified form 


Create a response that is 
mostly appropriate for the 
topic, purpose, and audience 


Create an appropriate response 
to the topic, purpose and 
audience 


Present a few ideas relevant to 
the topic, purpose, and 
audience and show some 
evidence of thinking and 
writing approaches that 
support development of ideas 


Present some ideas relevant to 
the topic, purpose, and 
audience and show some 
evidence of thinking and 
writing approaches that 
support development of ideas 


Present ideas that are relevant 
to the topic, purpose, and 
audience and show evidence 
of thinking and writing 
approaches that support 
development of ideas 


Use a few supporting details 
relevant to the topic, purpose, 
and audience 


Use some supporting details 
that are mostly relevant to the 
topic, purpose, and audience 


Use supporting details relevant 
to the topic, purpose, and 
audience 


Create a response whose 
organizational structure shows 
a little evidence of appropriate 
thinking and writing 
approaches 


Create a response whose 
organizational structure shows 
some evidence of appropriate 
thinking and writing 
approaches 


Create a response that usually 
focuses on the topic and 
whose organizational structure 
shows some evidence of 
appropriate thinking and 
writing approaches 
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Exhibit 6.1 (continued). Preliminary Basic Achievement Level Descriptions 



Grade 4 


Grade 8 


Grade 12 


Students performing at the Basic level should be able to: 


Some sentences are correct 
and the response uses a little 
sentence variety 


Most sentences are correct and 
the response uses some 
sentence variety 


Sentences are correct and the 
response uses some sentence 
variety as appropriate to 
communicate relationships 
among ideas 


Use word choice that is mostly 
clear and appropriate for the 
topic, purpose, and audience 


Use word choice that is 
usually clear and appropriate 
for the topic, purpose, and 
audience 


Use word choice that is clear 
and appropriate for the topic, 
purpose, and audience 


Use voice and tone that show 
some understanding of what is 
appropriate for the topic, 
purpose, and audience 


Use voice and tone that are 
mostly appropriate for the 
topic, purpose, and audience 


Use voice and tone that are 
appropriate for the topic, 
purpose, and audience 


Use grammar, usage, and 
mechanics that are mostly 
correct but with distracting 
errors that may occasionally 
impede understanding 


Use grammar, usage, and 
mechanics that are mostly 
correct but with some 
distracting errors that may 
occasionally impede 
understanding 


Use grammar, usage, and 
mechanics that are mostly 
correct but with a few 
distracting errors that may 
occasionally impede 
understanding 
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Exhibit 6.2. Preliminary Proficient Achievemenl 


Level Descriptions 


Grade 4 


Grade 8 


Grade 12 


Students performing at the Proficient level should be able to: 


Create a somewhat competent 
response to the topic, purpose, 
audience, and specified form 


Create a mostly competent 
response to the topic, purpose, 
and audience 


Create a competent response 
to the topic, purpose, and 
audience 


Present some ideas that are 
relevant to the topic, purpose, 
and audience and show some 
evidence of effective thinking 
and writing approaches that 
support development of ideas 


Present ideas that support the 
topic, purpose, and audience 
and show evidence of 
effective thinking and writing 
approaches that support 
development of ideas 


Present some ideas that 
support the topic, puipose, and 
audience and show consistent 
evidence of effective thinking 
and writing approaches that 
support development of ideas 


Use supporting details that are 
mostly relevant and effective 
to develop ideas 


Use supporting details that are 
relevant and effective to 
develop ideas 


Use specific and relevant 
supporting details to develop 
ideas 


Create a response using 
logical grouping of ideas and 
with some thinking and 
writing approaches that are 
relevant for the topic, purpose, 
and audience 


Create a response using 
logical grouping of ideas and 
thinking and writing 
approaches that are relevant to 
the topic, purpose, and 
audience 


Create a response that is 
organized logically, using 
effective and relevant thinking 
and writing approaches to 
support a logical progression 
of ideas 


Most sentences are controlled 
and there is some sentence 
variety, as appropriate, for the 
topic, purpose, and audience 


Use sentences that are usually 
controlled and that are varied, 
as appropriate, for the topic, 
purpose, and audience 


Use sentences that are 
controlled and that are varied, 
as appropriate, for the topic, 
purpose, and audience 


Use word choice that is 
sometimes specific and that is 
appropriate to communicate 
relationships among ideas 


Use word choice that is mostly 
specific and that is appropriate 
to communicate relationships 
among ideas 


Use some specific and precise 
word choice that is appropriate 
to communicate relationships 
among ideas 


Use voice and tone that are 
mostly controlled and 
appropriate for the topic, 
purpose, and audience 


Use voice and tone that are 
usually controlled and 
appropriate for the topic, 
purpose, and audience 


Use voice and tone that are 
controlled and appropriate for 
the topic, purpose, and 
audience 


Use grammar, usage, and 
mechanics that are mostly 
correct but with some 
distracting errors 


Use grammar, usage, and 
mechanics that are mostly 
correct but with some 
distracting errors 


Use grammar, usage, and 
mechanics that are mostly 
correct but with a few 
distracting errors 
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Exhibit 6.3. Preliminary Advanced Achievemen 


Level Descriptions 


Grade 4 


Grade 8 


Grade 12 


Students performing at the Advanced level should be able to: 


Create an effective response to 
the topic, purpose, audience, 
and specified form 


Create a thoughtful and 
effective response to topic, 
purpose, and audience 


Create an insightful response 
to the topic, purpose, and 
audience 


Present clear ideas that are 
relevant to the topic, purpose, 
and audience, and show 
evidence of relevant thinking 
and writing approaches that 
support development of ideas 


Present clear and effective 
ideas to support the topic, 
purpose, and audience, and 
provide clear evidence of 
relevant and effective thinking 
and writing approaches that 
support development of ideas 


Present thoughtful and 
complex ideas to support the 
topic and effectively use 
approaches for thinking and 
writing that support 
development of ideas 


Create a response that is 
organized effectively to show 
a somewhat logical 
progression of ideas and some 
evidence of relevant 
approaches for thinking and 
writing 


Create a response that is 
organized effectively to show 
a mostly logical progression of 
ideas and evidence of relevant 
approaches for thinking and 
writing 


Create a response with an 
organizational structure that 
has a clear and effective 
logical progression of ideas 
and that reflects effective use 
of relevant thinking and 
writing approaches 


Most sentences are well 
controlled and varied, as 
appropriate, to communicate 
relationships among ideas 


Sentences are well controlled 
and varied, as appropriate, to 
communicate relationships 
among ideas 


Use sentences that are well 
controlled and varied, as 
appropriate, to communicate 
relationships among ideas 


Use word choice that is 
specific and appropriate for 
the topic, purpose, and 
audience 


Use specific and mostly 
precise word choice 
appropriate for the topic, 
purpose, and audience 


Use specific and precise word 
choice appropriate for the 
topic, purpose, and audience 


Use voice and tone that are 
usually controlled for the 
topic, purpose, and audience 


Use voice and tone that are 
well controlled for the topic, 
purpose, and audience 


Use voice and tone that are 
well controlled for the topic, 
purpose, and audience 


Demonstrate good control of 
grammar, usage, and 
mechanics to communicate to 
the reader 


Demonstrate good control of 
grammar, usage, and 
mechanics and use some of 
these elements to enhance the 
presentation of ideas 


Demonstrate strong command 
of grammar, usage, and 
mechanics and use these 
elements to enhance the 
presentation of ideas 
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Appendix E 

NAEP Writing Special Study 



To improve the quality of the NAEP assessment, and to gain maximum information on 
student achievement, NAEP coordinates special studies on framework topics and issues. 
For the 2011 NAEP Writing, one special study has been proposed. 

STUDY: GRADE 4 COMPUTER-BASED WRITING 



Purpose 

This special study seeks to frame trends in computer support — both in the accessibility of 
computers in all schools and instruction on the computers — in order to achieve a greater 
understanding of how grade 4 students compose on computers. The study will also 
address the expectations for computer proficiency and the role of computers in instruction 
at the middle school level. This information will provide a clearer research base for future 
computer-based assessment practices by NAEP. 

Rationale 

The 2011 NAEP Writing will assess computer-based writing with some commonly 
available tools at grades 8 and 12. The 2011 NAEP Writing Framework recommends that 
a computer-based platform also be implemented at grade 4 during the tenure of this 
framework — by 2019, if feasible. 

Prior to any implementation of a computer-based writing assessment at grade 4, NAEP 
would benefit from additional research on the computer use of elementary students, 
particularly on keyboarding pedagogy, levels of computer literacy among elementary 
students, and computer use within the composition process. Exploration of these issues 
will inform the discussion of when a computer-based writing assessment at grade 4 will 
be warranted. 

Research interests for this study comprise five categories: keyboarding experience, 
writing instruction and computers, computer tools and applications, assessment validity, 
and computer proficiency needed when students reach middle school. 
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Research Questions 

1. Opportunities for computer-based writing 

• How frequently do grade 4 students use computers and for what purposes? 

• How often and in what ways does writing instruction at grade 4 involve 
computers? 

• What parts of the writing process are computer-based? 

2. Keyboarding experience 

• Are there significant differences among groups with regard to keyboarding 
proficiency (e.g., gender, race/ethnicity, socio-economic status, type of school 
[e.g., large urban/suburban/rural, etc.])? 

• What factors impact differences in keyboarding proficiency (e.g., access to 
computers, when instruction in keyboarding begins)? 

• When are keyboarding skills first taught? When do elementary students have the 
finger size, manual dexterity, attention span, and hand-eye coordination skills 
needed to begin keyboarding instruction? What is the interaction between 
keyboarding instruction and developing cursive handwriting skills? 

• How many words per minute do students need to type in order to successfully 
compose on the computer for a time-limited writing? 

• In what ways is access to computers outside of schools a factor in grade 4 
students’ proficiency with computers? 

3. Platform for computer-based writing at grade 4 

• What applications and tools are most commonly available to grade 4 students for 
computer-based writing? Which tools are most commonly used? Are tools a 
positive or negative influence on the development of ideas, organization of ideas, 
and/or language facility? 

4. Proficiency grade 4 students should have 

• What proficiency should grade 4 students have with computer-based writing? 
What is expected on states’ grade level assessments? What skills are needed for 
success in middle school? 

5. Accuracy of assessment 

• What mode (handwritten or computer-based) is most likely to support an accurate 
assessment of grade 4 students’ writing ability? 

As a result of the Governing Board May 2010 decision supporting a computer-based 

NAEP Writing Assessment at grade 4, a significant amount of research will be needed in 

the assessment development, which may include one or more studies addressing the 

research questions in this special study. 
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Appendix G 



Organizations Contacted to Review Initial 2011 NAEP 

Writing Framework Recommendations 



Organization 


Conference 
or Focus 
Group 
Session 


Listserv, Web 
Posting or 
Mass E-mail 


Achieve, Inc. 




Yes 


ACT Item Writers — Educators from all subjects 




Yes 


Alliance for Excellent Education 




Multiple 
advisory boards 
and AEE 
newsletter 


American Federation of Teachers (AFT) 




Yes 


Assembly of State Coordinators of ELA (ASCELA) 




Yes 


Association for Supervision and Curriculum Directors (ASCD) 


Yes 


Yes 


Business Coalition for Educational Excellence (BCEE) 


Yes 




Business Round Table 




Yes 


California Association of Teachers of English (CATE) 




Yes 


Chicago Area Reading Coaches 


Yes 




The Coca-Cola Company 




Yes 


Conference on College Composition and Communication 
(CCCC) 


Yes 


Yes 


Conference on English Leadership (CEL) 


Yes 




Council for American Private Education (CAPE) 




Yes 


Council of Chief State School Officers (CCSSO) 


Yes 




Council of Great City Schools (CGCS) 


Yes 




Eastern Community College Social Science Association 




Yes 


Education Information Management Advisory Consortium 
(EIMAC) 




Yes 


Education Service Agency 




Yes 


Houston ESL Supervisors and Lead Teachers 




Yes 


International Reading Association (IRA) 


Yes 


Yes 


Iowa Guidance Counselors 




Yes 


Los Angeles Unified School District 




Yes 


Midcontinent Regional Education Lab (McREL) 




Yes 


Minnesota District Assessment Coordinators 




Yes 


National Council of Teachers of English (NCTE) 




Yes 


National Association of Elementary School Principals 
(NAESP) 


Yes 


Yes 


National Association of Secondary School Principals (NASSP) 




Yes 
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Organization 


Conference 
or Focus 
Group 
Session 


Listserv, Web 
Posting or 
Mass E-mail 


National Association of State Boards of Education (NASBE) 




Yes 


National Association of State Directors of Special Education 
(NASDSE) 




Yes 


National Association of Workforce Development Professionals 
(NAWDP) 


Yes 


Yes 


National Catholic Education Association & Diocese 
Connections 


Yes 


Yes 


National Conference on Research in Language and Literacy 
(NCRLL) 




Yes 


National Council of Teachers of English (NCTE) 


Yes 




National Middle School Association 




Yes 


National Parent Teacher Association (PTA) 




Yes 


National Reading Conference 




Yes 


National School Board Association (NSBA) 




Yes 


National Science Teachers Association (NSTA) 




Yes 


National Writing Project (NWP) — directors and teacher- 
consultants, technology experts, urban directors and teacher 
consultants, English language learner experts 


Yes (x 5) 




Northern Virginia Community College System 




Yes 


Recent high school graduates 




Yes 


Richland College of the Dallas County 




Yes 


Society for Human Resources Management (SHRM) 


Yes (x 4) 




South Dakota State Department of Education 




Yes 


Teachers of English to Speakers of Other Languages (TESOL) 


Yes 


Yes 


V arious technology and writing listservs 




Yes 


Washington State Writing Assessment Leaders 




Yes 



Additional Organizations Contacted: 

Air Force Academy 
Air Force ROTC 
Army ROTC 

Center for Applied Linguistics (CAL) 

The Citadel 
Education Trust 
Marines ROTC 

National Rural Education Association (NREA) 
Naval Academy 
Navy ROTC 

United States Military Academy at West Point 
Virginia Military Institute 
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